Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-18360

Migrate storage to new Binary Tuple format

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0
    • None

    Description

      The Binary Tuple Format was introduced in IEP-92 as replacement of IEP-54 Binary Row. However, the Ignite's core is still operating rows in BinaryRow format. Let's start the migration to the new format.

      Under current ticket it's proposed to migrate the storage only, as these changes looks predictable and isolated, whereas it allows to start migration in SQL engine as well.

      The migration plan is as follow:

      1. Introduce new entity named TableRow.
        Unlike the indexes, the table may evolve over time, thus schema of the binary tuple may change as well. To make the storage a schema agnostic, it's proposed to store only the version of the schema and a ByteBuffer representing the data in the Binary Tuple format.
      2. Migrate all table-related storage interfaces and PartitionListener to the new format
      3. Make conversion in the PartitionReplicaListener from BinaryRow to TableRow and vice versa on the edge Replica-Storage integration (in the context of this issue I consider the PartitionListener as part of the storage).
        PartitionReplicaListener (PRL) seems to be a good choice to make such a conversion because of 1) current implementation of row comparison for deleteExact operation can't properly handle the schema changes, and thus the notion of the schema has to be brought to this level, and 2) PRL is build upon async calls, so it will be easy to incorporate awaiting of schema in case the node stales a bit.
        NB: during conversion, I would prefer to restore the logical order of the columns and build the tuple in that order, rather than keep the physical order of BinaryRow

      Attachments

        Issue Links

          Activity

            People

              vpakhnushev Vadim Pakhnushev
              korlov Konstantin Orlov
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h
                  4h