Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-28341

Iceberg: Change Major QB Full Table Compaction to compact partitions in parallel

    XMLWordPrintableJSON

Details

    Description

      Currently, Major compaction compacts a whole table in one step. If a table is partition and has a lot of data this operation can take a lot of time and it risks getting write conflicts at the commit stage. This can be improved to work partition by partition. Also, for each partition it will create one snapshot instead of 2 snapshots (truncate+IOW) created now when compacting the whole table in one step.

      Attachments

        Issue Links

          Activity

            People

              difin Dmitriy Fingerman
              difin Dmitriy Fingerman
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: