Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-10004

Delete unnecessary flush in Lucene90CompressingStoredFieldsWriter.copyChunks() to reduce dirty chunks

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Not A Problem
    • 8.8.2
    • None
    • core/codecs
    • None
    • New

    Description

      In CompressingStoredFieldsWriter.merge(): if the segment meet the following conditions:

      else if (matchingFieldsReader.getCompressionMode() == compressionMode && 
                       matchingFieldsReader.getChunkSize() == chunkSize && 
                       matchingFieldsReader.getPackedIntsVersion() ==PackedInts.VERSION_CURRENT &&
                       liveDocs == null &&
                       !tooDirty(matchingFieldsReader)) { 
             ......
             // flush any pending chunks
              if (numBufferedDocs > 0) {
                flush();
                numDirtyChunks++; // incomplete: we had to force this flush
              }
             ......
      }
      

      We will copy the the all chunk to the new fdt, before copying the chunk, we will flush the buffer docs if numBufferedDocs >0, but the flush is unnecessary.

      The bufferedDocs in memory have nothing to do with copyChunk. We just need to ensure that it will be flush at the end of merge(In finish()).

      Attachments

        Activity

          People

            Unassigned Unassigned
            kkewwei kkewwei
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: