Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10805

Speed up KeyValueHeap.next() a bit

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.98.1, 0.99.0, 0.94.19, 0.96.3
    • None
    • None
    • Reviewed

    Description

      See discussion on HBASE-9969.
      This was brought up by mcorgan. Even though I didn't believe him first.

            KeyValueScanner topScanner = this.heap.peek();
            if (topScanner == null ||
                this.comparator.compare(kvNext, topScanner.peek()) >= 0) {
              this.heap.add(this.current);
              this.current = pollRealKV();
            }
      

      We already have the invariant everywhere this.current always has a real KV polled. So adding current back to the heap followed by pollRealKV() is a no-op if this.current is the only scanner anyway.

      This can be changed to:

            ...
            if (topScanner != null &&
                this.comparator.compare(kvNext, topScanner.peek()) >= 0) {
            ...
      

      With 20m rows, 5 cols each, everything in the cache, fully compacted - i.e. one HFile per store, a scan that filters everything at the server through all 100m KVs takes 15.1s without the change and 13.3s with it, so a 12% improvement.

      Attachments

        1. 10805-trunk.txt
          0.8 kB
          Lars Hofhansl

        Activity

          People

            larsh Lars Hofhansl
            larsh Lars Hofhansl
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: