Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
None
-
None
-
ghx-label-12
Description
The following job failed due to out of memory:
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/12588 (please click on "Don't keep this build forever" once this issue is resolved)
Relevant log lines:
02:33:42 Loading nested orc data (logging to /home/ubuntu/Impala/logs/data_loading/load-nested.log)... 02:35:39 FAILED (Took: 1 min 57 sec) 02:35:39 '/home/ubuntu/Impala/testdata/bin/load_nested.py -t tpch_nested_orc_def -f orc/def' failed. Tail of log: 02:35:39 2020-11-11 02:35:06,225 INFO:load_nested[348]:Executing: 02:35:39 02:35:39 CREATE EXTERNAL TABLE supplier 02:35:39 STORED AS orc 02:35:39 TBLPROPERTIES('orc.compress' = 'ZLIB','external.table.purge'='TRUE') 02:35:39 AS SELECT * FROM tmp_supplier 02:35:39 Traceback (most recent call last): 02:35:39 File "/home/ubuntu/Impala/testdata/bin/load_nested.py", line 415, in <module> 02:35:39 load() 02:35:39 File "/home/ubuntu/Impala/testdata/bin/load_nested.py", line 349, in load 02:35:39 hive.execute(stmt) 02:35:39 File "/home/ubuntu/Impala/tests/comparison/db_connection.py", line 206, in execute 02:35:39 return self._cursor.execute(sql, *args, **kwargs) 02:35:39 File "/home/ubuntu/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py", line 331, in execute 02:35:39 self._wait_to_finish() # make execute synchronous 02:35:39 File "/home/ubuntu/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py", line 413, in _wait_to_finish 02:35:39 raise OperationalError(resp.errorMessage) 02:35:39 impala.error.OperationalError: Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1605060173780_0039_2_00, diagnostics=[Task failed, taskId=task_1605060173780_0039_2_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Container container_1605060173780_0039_01_000002 finished with diagnostics set to [Container failed, exitCode=-104. [2020-11-11 02:35:11.768]Container [pid=16810,containerID=container_1605060173780_0039_01_000002] is running 7729152B beyond the 'PHYSICAL' memory limit. Current usage: 1.0 GB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used. Killing container.
Attachments
Issue Links
- is duplicated by
-
IMPALA-10669 Loading nested ORC data is flaky during Docker-based tests
- Resolved
- is related to
-
IMPALA-10503 testdata load hits hive memory limit errors during hive inserts
- Resolved
-
IMPALA-10669 Loading nested ORC data is flaky during Docker-based tests
- Resolved
- relates to
-
HIVE-26346 Default Tez memory limits occasionally result in killing container
- Open