Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
4.0.0
-
None
Description
if ((deltaCount + (dir.getBaseDirectory() == null ? 0 : 1)) + origCount <= 1) { LOG.debug("Not compacting {}; current base is {} and there are {} deltas and {} originals", sd.getLocation(), dir .getBaseDirectory(), deltaCount, origCount); return; }
Is problematic.
Suppose you have 1 delta file from streaming ingest: delta_11_20 where txnid:13 was aborted. The code above will not rewrite the delta (which drops anything that belongs to the aborted txn) and transition the compaction to "ready_for_cleaning" state which will drop the metadata about the aborted txn in markCleaned(). Now aborted data will come back as committed.
Attachments
Attachments
Issue Links
- relates to
-
HIVE-9995 ACID compaction tries to compact a single file
- Closed
- links to