Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-5898

Consider double-checked locking for block cache lock

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.94.1
    • 0.94.3, 0.95.0
    • Performance
    • None

    Description

      Running a workload with a high query rate against a dataset that fits in cache, I saw a lot of CPU being used in IdLock.getLockEntry, being called by HFileReaderV2.readBlock. Even though it was all cache hits, it was wasting a lot of CPU doing lock management here. I wrote a quick patch to switch to a double-checked locking and it improved throughput substantially for this workload.

      Attachments

        1. hbase-5898.txt
          2 kB
          Todd Lipcon
        2. 5898-TestBlocksRead.txt
          3 kB
          Ted Yu
        3. HBASE-5898-0.patch
          6 kB
          Elliott Neil Clark
        4. HBASE-5898-1.patch
          6 kB
          Elliott Neil Clark
        5. HBASE-5898-1.patch
          6 kB
          Michael Stack
        6. 5898-v2.txt
          6 kB
          Lars Hofhansl
        7. 5898-v3.txt
          8 kB
          Lars Hofhansl
        8. 5898-v4.txt
          30 kB
          Lars Hofhansl
        9. 5898-v4.txt
          30 kB
          Lars Hofhansl
        10. 5898-0.94.txt
          30 kB
          Lars Hofhansl

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HBASE-7336 HFileBlock.readAtOffset does not work well with multiple threads

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-5987 HFileBlockIndex improvement

          • Major - Major loss of function.
          • Closed
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-7160 Improve IdLock and remove its minor defects

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              tlipcon Todd Lipcon
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: