Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21543

when using FIFO compaction, I found sst being deleted on the first checkpoint

    XMLWordPrintableJSON

Details

    Description

      2021/03/01-18:51:01.202049 7f59042fc700 (Original Log Time 2021/03/01-18:51:01.200883) [/compaction/compaction_picker_fifo.cc:107] [_timer_state/processing_user-timers] FIFO compaction: picking file 1710 with creation time 0 for deletion

       

      the configuration is like 

      currentOptions.setCompactionStyle(getCompactionStyle());
      currentOptions.setLevel0FileNumCompactionTrigger(8);
      // currentOptions.setMaxTableFilesSizeFIFO(MemorySize.parse("2gb").getBytes());
      CompactionOptionsFIFO compactionOptionsFIFO = new CompactionOptionsFIFO();
      compactionOptionsFIFO.setMaxTableFilesSize(MemorySize.parse("8gb").getBytes());
      compactionOptionsFIFO.setAllowCompaction(true);

       

      the rocksdb version is 

      <dependency>
      <groupId>io.github.myasuka</groupId>
      <artifactId>frocksdbjni</artifactId>
      <version>6.10.2-ververica-3.0</version>
      </dependency>

       

      I think the problem is caused by tableproperty is lost by snapshot. Can any one suggest how i can skip this problem?

      Attachments

        1. image-2021-03-03-13-09-01-695.png
          66 kB
          xiaogang zhou
        2. image-2021-03-03-11-35-11-458.png
          250 kB
          xiaogang zhou
        3. LOG (2)
          38.76 MB
          xiaogang zhou

        Activity

          People

            Unassigned Unassigned
            zhoujira86 xiaogang zhou
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: