Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1400

Improve rowset compaction policy to consider merging small DRSs

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Done
    • None
    • 1.9.0
    • None
    • None

    Description

      We see some small table with light write load generate lot's of small DRS(~1MB), since those DRSes do not overlap much, they don't get the chance to be compacted, generating lot of very small files/blocks. So:

      1. Compaction solution value should consider benefits of merging small DRS
      2. Every 2 min flushing MRS(small or large) seems suboptimal, maybe flushing small MRS should have "lower priority" than rowset compaction with higher solution value?

      Attachments

        Issue Links

          Blocked

          Bug - A problem which impairs or prevents the functions of the product. KUDU-2648 compaction does not run

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. KUDU-2318 Kudu flushes aggressively even if it can't relieve memory pressure.

          • Major - Major loss of function.
          • Open
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. KUDU-1625 Schedule compaction on rowsets with high percentage of deleted data

          • Major - Major loss of function.
          • Open

          Activity

            People

              wdberkeley William Berkeley
              decster Binglin Chang
              Votes:
              1 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: