Details
-
Task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
In iceberg world, we should not delete file in a dataset directly, so re-compaction should not delete the old daily data, instead it should write to a new folder. This ticket is to add a config to enable re-compaction write to a new folder instead of overwrite the existing folder. After the configure is enabled, the daily data directory will be /daily/yyyy/mm/dd/compaction_{executionCount}