Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
Impala 4.1.0
-
None
-
ghx-label-9
Description
When the files of n Iceberg table are dropped then a DROP TABLE will result in an error while the table will still show up in SHOW TABLES
Here are the steps to repro:
1) Run from Impala-shell
DROP DATABASE IF EXISTS `drop_incomplete_table` CASCADE; CREATE DATABASE `drop_incomplete_table`; CREATE TABLE drop_incomplete_table.iceberg_tbl (i int) stored as iceberg tblproperties('iceberg.catalog'='hadoop.catalog', 'iceberg.catalog_location'='/test-warehouse/drop_incomplete_table'); INSERT INTO drop_incomplete_table.iceberg_tbl VALUES (1), (2), (3);
2) Drop the folder of the table with hdfs dfs
hdfs dfs -rm -r hdfs://localhost:20500/test-warehouse/drop_incomplete_table
3) Try to drop the table from Impala-shell
DROP TABLE drop_incomplete_table.iceberg_tbl;
This results in the following error:
ERROR: TableNotFoundException: Table 'drop_incomplete_table.iceberg_tbl' does not exist in Iceberg catalog.
While table is still there in show tables output even after an invalidate metadata.
Note, it's important for the repro to execute some SQL against the newly created table to load it in Impala. In this case I used an INSERT INTO but e.g. an ALTER TABLE would also be good. Apparently, when the table is "incomplete" (this is the state right after running CREATE TABLE) this works fine but not if the table is loaded.
The suspicious part of code is in StmtMetadataLoader.loadTables() and getMissingTables() where there is a distinction between loaded and Incomplete tables.
https://github.com/apache/impala/blob/2f74e956aa10db5af6a7cdc47e2ad42f63d5030f/fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java#L196
Attachments
Issue Links
- is cloned by
-
IMPALA-11503 Dropping files of Iceberg table in HiveCatalog will cause DROP TABLE to fail
- Open
- relates to
-
IMPALA-11503 Dropping files of Iceberg table in HiveCatalog will cause DROP TABLE to fail
- Open