Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
Impala 2.11.0, Impala 2.12.0
-
None
-
ghx-label-7
Description
Stack trace:
query_test/test_observability.py:123: in test_profile_fragment_instances assert results.runtime_profile.count("HDFS_SCAN_NODE") == 12 E assert 11 == 12 E + where 11 = <built-in method count of str object at 0x68bd2f0>('HDFS_SCAN_NODE') E + where <built-in method count of str object at 0x68bd2f0> = 'Query (id=ae4cee91aafc5c6c:11b545c600000000):\n DEBUG MODE WARNING: Query profile created while running a DEBUG buil...ontextSwitches: 0 (0)\n - TotalRawHdfsReadTime(*): 5s784ms\n - TotalReadThroughput: 17.33 MB/sec\n'.count E + where 'Query (id=ae4cee91aafc5c6c:11b545c600000000):\n DEBUG MODE WARNING: Query profile created while running a DEBUG buil...ontextSwitches: 0 (0)\n - TotalRawHdfsReadTime(*): 5s784ms\n - TotalReadThroughput: 17.33 MB/sec\n' = <tests.beeswax.impala_beeswax.ImpalaBeeswaxResult object at 0x6322e10>.runtime_profile
Query:
with l as (select * from tpch.lineitem UNION ALL select * from tpch.lineitem) select STRAIGHT_JOIN count(*) from (select * from tpch.lineitem a LIMIT 1) a join (select * from l LIMIT 2000000) b on a.l_orderkey = -b.l_orderkey;
Summary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail -------------------------------------------------------------------------------------------------------------------- 05:AGGREGATE 1 0.000ns 0.000ns 1 1 28.00 KB 10.00 MB FINALIZE 04:HASH JOIN 1 15.000ms 15.000ms 0 1 141.06 MB 17.00 MB INNER JOIN, BROADCAST |--08:EXCHANGE 1 4s153ms 4s153ms 2.00M 2.00M 0 0 UNPARTITIONED | 07:EXCHANGE 1 3s783ms 3s783ms 2.00M 2.00M 0 0 UNPARTITIONED | 01:UNION 3 17.000ms 28.001ms 3.03M 2.00M 0 0 | |--03:SCAN HDFS 3 0.000ns 0.000ns 0 6.00M 0 176.00 MB tpch.lineitem | 02:SCAN HDFS 3 6s133ms 6s948ms 3.03M 6.00M 24.02 MB 176.00 MB tpch.lineitem 06:EXCHANGE 1 5s655ms 5s655ms 1 1 0 0 UNPARTITIONED 00:SCAN HDFS 3 4s077ms 6s207ms 2 1 16.05 MB 176.00 MB tpch.lineitem a
Plan:
---------------- Max Per-Host Resource Reservation: Memory=17.00MB Per-Host Resource Estimates: Memory=379.00MB F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=27.00MB mem-reservation=17.00MB PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B | 05:AGGREGATE [FINALIZE] | output: count(*) | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB | tuple-ids=7 row-size=8B cardinality=1 | 04:HASH JOIN [INNER JOIN, BROADCAST] | hash predicates: a.l_orderkey = -1 * l_orderkey | fk/pk conjuncts: assumed fk/pk | runtime filters: RF000[bloom] <- -1 * l_orderkey | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB | tuple-ids=0,4 row-size=16B cardinality=1 | |--08:EXCHANGE [UNPARTITIONED] | | mem-estimate=0B mem-reservation=0B | | tuple-ids=4 row-size=8B cardinality=2000000 | | | F05:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=0B mem-reservation=0B | 07:EXCHANGE [UNPARTITIONED] | | limit: 2000000 | | mem-estimate=0B mem-reservation=0B | | tuple-ids=4 row-size=8B cardinality=2000000 | | | F04:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 | Per-Host Resources: mem-estimate=176.00MB mem-reservation=0B | 01:UNION | | pass-through-operands: all | | limit: 2000000 | | mem-estimate=0B mem-reservation=0B | | tuple-ids=4 row-size=8B cardinality=2000000 | | | |--03:SCAN HDFS [tpch.lineitem, RANDOM] | | partitions=1/1 files=1 size=718.94MB | | stored statistics: | | table: rows=6001215 size=718.94MB | | columns: all | | extrapolated-rows=disabled | | mem-estimate=176.00MB mem-reservation=0B | | tuple-ids=3 row-size=8B cardinality=6001215 | | | 02:SCAN HDFS [tpch.lineitem, RANDOM] | partitions=1/1 files=1 size=718.94MB | stored statistics: | table: rows=6001215 size=718.94MB | columns: all | extrapolated-rows=disabled | mem-estimate=176.00MB mem-reservation=0B | tuple-ids=2 row-size=8B cardinality=6001215 | 06:EXCHANGE [UNPARTITIONED] | limit: 1 | mem-estimate=0B mem-reservation=0B | tuple-ids=0 row-size=8B cardinality=1 | F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 Per-Host Resources: mem-estimate=176.00MB mem-reservation=0B 00:SCAN HDFS [tpch.lineitem a, RANDOM] partitions=1/1 files=1 size=718.94MB runtime filters: RF000[bloom] -> a.l_orderkey stored statistics: table: rows=6001215 size=718.94MB columns: all extrapolated-rows=disabled limit: 1 mem-estimate=176.00MB mem-reservation=0B tuple-ids=0 row-size=8B cardinality=1 ----------------
Attachments
Attachments
Issue Links
- breaks
-
IMPALA-6484 Crash in impala::RuntimeProfile::SortChildren
- Resolved
- is blocked by
-
IMPALA-2990 Coordinator should timeout and cancel queries with unresponsive / stuck executors
- Resolved
-
IMPALA-5384 Simplify coordinator locking protocol
- Resolved
- is duplicated by
-
IMPALA-6535 test_profile_fragment_instances failure: incomplete query profile
- Resolved
-
IMPALA-6960 TestMtDopParquet.test_parquet_filtering flaky
- Resolved
- is related to
-
IMPALA-341 Remote profiles may be ignored by coordinator if query has a limit
- Resolved
-
IMPALA-6960 TestMtDopParquet.test_parquet_filtering flaky
- Resolved