Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-15434

Implement cursor for partitions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha3
    • None

    Description

      In order to unblock sql activities in required to have partition scanning logic. Simple approach would be similar to meta storage range command: cursors  on the client side with corresponding raft commands that will manage server side storage cursors. However given approach has some disadvantages, for example, cursor's hasNext(), next(), close() etc are blocking by design so it might lead (and as it's seen with Meta storage range will lead) to a deadlock if response retrieval logic is blocked by synchronous get on hasNext or similar. In order to overcome given issue we might use unblocking cursors based on reactive flow model. It actually means that client side range/scan should provide Publisher instance*.*

      Implementation details:

      Within InternalTable class new method was introduced:

      /**
       * Scans given partition, providing {@link Publisher<BinaryRow>} that reflectively notifies about partition rows.
       * @param p The partition.
       * @param tx The transaction.
       * @return {@link Publisher<BinaryRow>} t{@link Publisher<BinaryRow>} that reflectively notifies about partition rows.
       */
      @NotNull Publisher<BinaryRow> scan(int p, @Nullable Transaction tx);
      

      Core publisher logic is implemented inside:

      InternalTableImpl.PartitionScanPublisher.PartitionScanSubscription#PartitionScanSubscription
      

      Please pay attention that inner batching logic is not implemented. PartitionScanSubscription will request same amount of entries from Storage as it was specified within subscription.request(). Seems that it doesn't worth to implement it right now cause event reactive model wasn't approved yet.

      Three new partition raft commands and corresponding handlers were added:

      • ScanInitCommand
      • ScanRetrieveBatchCommand
      • ScanCloseCommand

      All given commands are write ones, so it'll lead to full scan replication over raft nodes. On the one hand such approach leads to transparent remapping logic that eases sql quey processing logic in case of raft leadership changes, on the other hand it requires more time for replication and more space within raft log for such commands. In future such strong restrictions might be be weakened.

       

      Attachments

        Issue Links

          Activity

            People

              alapin Alexander Lapin
              alapin Alexander Lapin
              Konstantin Orlov Konstantin Orlov
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 5h 20m
                  5h 20m