Uploaded image for project: 'Apache IoTDB'
  1. Apache IoTDB
  2. IOTDB-918

[Distributed] Bind TsFiles with log index to accelerate snapshots

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • None
    • Core/Cluster

    Description

      During snapshot catch-up, the leader will collect information of all TsFiles as RemoteTsFileResources, and send them to the followers. The follower will check if it has the file and decide whether to pull it from the leader.

      To be smarter, the leader may be able to know that a TsFile of a certain version only contains data precedes to a certain log index, and the leader can avoid sending files that would be surely on the receiver side.

      For example, when a TsFile `3.tsfile` is closed and the current committed log index is 100, the leader knows for sure that the file will not contain data corresponding to logs whose indexes are more than 100. When a follower whose committed log index exceeds 100 applies for a snapshot, the leader can skip `3.tsfile` since the follower must have it.

      Attachments

        Issue Links

          Activity

            People

              jt2594838 Tian Jiang
              jt2594838 Tian Jiang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: