Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
3.1.0, 3.1.2
Description
Hive table 'aa.bb_item' is corrupt. Found sub-directory 'abfs://aabb-hive-data@gen2hiveaabb.dfs.core.windows.net/prod-data/aa.db/bb_item/part_created_on=202105/base_0004042/base_0004042' in bucket directory for partition: part_created_on=202105 at io.prestosql.plugin.hive.BackgroundHiveSplitLoader.loadPartition(BackgroundHiveSplitLoader.java:543) at io.prestosql.plugin.hive.BackgroundHiveSplitLoader.loadSplits(BackgroundHiveSplitLoader.java:325) at io.prestosql.plugin.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:254) at io.prestosql.plugin.hive.util.ResumableTasks$1.run(ResumableTasks.java:38) at io.prestosql.$gen.Presto_347____20210615_143054_2.run(Unknown Source) at io.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:80) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829);
Why it happens:
Multiple compaction jobs for the same transactions can be triggered if the HMS gets restarted and the MR job is still in progress.
Attachments
Issue Links
- links to