Details
-
Task
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
None
-
None
-
None
Description
Sorted Index Storage currently uses BinaryRow as way to convert column values into byte arrays. This approach is not optimal for the following reasons:
- Data is stored in RocksDB and we can't use its native lexicographic comparator, we rely on a custom Java-based comparator that needs to de-serialize all columns in order to compare them. This is bad performance-wise, because Java-based comparators are slower and we need to extract all column values;
- Range scans can't use the prefix seek operation from RocksDB, because BinaryRow seralization is not stable: serialized prefix of column values will not be a prefix of the whole serialized row, because the format depends on columns being serialized;
- BinaryRow serialization is designed to store versioned row data and is overall badly suited to the Sorted Index purposes, its API usage looks awkward in this context.
We need to find a new serialization protocol that will (ideally) satisfy the following requirements:
- It should be comparable lexicographically;
- It should support null values;
- It should support variable length columns (though this requirement can probably be dropped);
- It should support both ascending and descending order for individual columns;
- It should support all data types that BinaryRow uses.
Attachments
Issue Links
- Dependent
-
IGNITE-16156 Byte ordered index keys.
- Resolved
- is part of
-
IGNITE-14925 Sorted indexes engine
- Resolved