Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
Impala 2.11.0
-
None
-
ghx-label-2
Description
Upgraded Impala cluster recently from 2.5 to 2.11 through CDH and observed that for some queries, memory estimates are very high now compared to pre upgrade days. Attached 1 such query whose memory estimates used to be 13-15 GB before started showing as 256 GB after the upgrade and not even entering the pool as it is directly getting rejected because of max pool size. As a temp fix, We have set mem limit for those queries and managing it for now.
In addition, also seeing different memory estimates for different users belonging to different pools for the same query. Does pool max size plays an any role in memory estimates? Surprisingly, observed this behaviour on 1st day of upgrade. Tried to reproduce now, but it is showing same memory estimates for different users.
Attachments
Attachments
Issue Links
- relates to
-
IMPALA-6460 More flexible memory-based admission control policies
- Resolved
-
IMPALA-6847 Consider adding workaround for high memory estimates in admission control
- Resolved