Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-36186

Speed up RocksDB close during manual compaction

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.0.0
    • 2.0.0
    • None

    Description

      After https://issues.apache.org/jira/browse/FLINK-26050, Flink RocksDB may schedule manual compaction asynchronously, but if a failover occurs at this time, RocksDB will need to wait for the manual comparison to complete before it can close. This may result in a very long time for task cancellation, affecting the time for task recovery.

      // After this function call, CompactRange() or CompactFiles() will not
      // run compactions and fail. Calling this function will tell outstanding
      // manual compactions to abort and will wait for them to finish or abort
      // before returning.
      virtual void DisableManualCompaction() = 0; 

      The solution is relatively simple. We can manually call DisabManulCompaction during db close to abort the running ManulCompaction, which can accelerate db close faster.

      Attachments

        Activity

          People

            Unassigned Unassigned
            mayuehappy Yue Ma
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: