Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
Impala 2.11.0
-
ghx-label-9
Description
Seen in an asf-master ASAN job at git hash 6b37a793a0af74ba01c67bb5ac7285af7c5c4d52:
04:15:21 ____________________ TestAlwaysFalseFilter.test_skip_split _____________________ 04:15:21 04:15:21 self = <test_always_false_filter.TestAlwaysFalseFilter object at 0x347a810> 04:15:21 cursor = <impala.hiveserver2.HiveServer2Cursor object at 0x3246310> 04:15:21 04:15:21 @SkipIfBuildType.not_dev_build 04:15:21 @pytest.mark.execute_serially 04:15:21 @CustomClusterTestSuite.with_args("--skip_file_runtime_filtering=true") 04:15:21 def test_skip_split(self, cursor): 04:15:21 """IMPALA-5789: Test that always false filter filters out splits when file-level 04:15:21 filtering is disabled. The filtering is not enabled in seq-based file formats.""" 04:15:21 cursor.execute("SET RUNTIME_FILTER_MODE=GLOBAL") 04:15:21 cursor.execute("SET RUNTIME_FILTER_WAIT_TIME_MS=10000") 04:15:21 query = """select STRAIGHT_JOIN * from alltypes inner join 04:15:21 (select * from alltypessmall where smallint_col=-1) v 04:15:21 on v.year = alltypes.year""" 04:15:21 # Manually iterate through file formats instead of creating a test matrix to prevent 04:15:21 # the cluster from restarting multiple times. 04:15:21 for table_suffix in ['_avro', '_rc', '_seq']: 04:15:21 cursor.execute("use functional" + table_suffix) 04:15:21 cursor.execute(query) 04:15:21 profile = cursor.get_profile() 04:15:21 assert re.search("Files rejected: [^0] \([^0]\)", profile) is None 04:15:21 assert re.search("Splits rejected: [^0] \([^0]\)", profile) is None 04:15:21 for table_suffix in ['', '_parquet']: 04:15:21 cursor.execute("use functional" + table_suffix) 04:15:21 cursor.execute(query) 04:15:21 profile = cursor.get_profile() 04:15:21 assert re.search("Files rejected: [^0] \([^0]\)", profile) is None 04:15:21 > assert re.search("Splits rejected: 8 \(8\)", profile) is not None 04:15:21 E assert None is not None 04:15:21 E + where None = <function search at 0x1076cf8>('Splits rejected: 8 \\(8\\)', 'Query (id=b1414c2cf5045bec:4e93cc3200000000):\n DEBUG MODE WARNING: Query profile created while running a DEBUG buil...ryContextSwitches: 7 (7)\n - TotalRawHdfsReadTime(*): 4.999ms\n - TotalReadThroughput: 0.00 /sec\n') 04:15:21 E + where <function search at 0x1076cf8> = re.search 04:15:21 04:15:21 custom_cluster/test_always_false_filter.py:53: AssertionError
Attachments
Issue Links
- is broken by
-
IMPALA-5789 Prune all partitions if an always-false runtime filter is received
- Resolved