Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-16105

Replace sorted index binary storage protocol

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • None

    Description

      Sorted Index Storage currently uses BinaryRow as way to convert column values into byte arrays. This approach is not optimal for the following reasons:

      1. Data is stored in RocksDB and we can't use its native lexicographic comparator, we rely on a custom Java-based comparator that needs to de-serialize all columns in order to compare them. This is bad performance-wise, because Java-based comparators are slower and we need to extract all column values;
      2. Range scans can't use the prefix seek operation from RocksDB, because BinaryRow seralization is not stable: serialized prefix of column values will not be a prefix of the whole serialized row, because the format depends on columns being serialized;
      3. BinaryRow serialization is designed to store versioned row data and is overall badly suited to the Sorted Index purposes, its API usage looks awkward in this context.

      We need to find a new serialization protocol that will (ideally) satisfy the following requirements:

      1. It should be comparable lexicographically;
      2. It should support null values;
      3. It should support variable length columns (though this requirement can probably be dropped);
      4. It should support both ascending and descending order for individual columns;
      5. It should support all data types that BinaryRow uses.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              apolovtcev Aleksandr Polovtsev
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: