Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17204 Make Off heap Bucket Cache default ON
  3. HBASE-19357

Bucket cache no longer L2 for LRU cache

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0-beta-1, 2.0.0
    • None
    • None
    • Incompatible change, Reviewed
    • Hide
      Removed cacheDataInL1 option for HCD
      BucketCache is no longer the L2 for LRU on heap cache. When BC is used, data blocks will be strictly on BC only where as index/bloom blocks are on LRU L1 cache.
      Config 'hbase.bucketcache.combinedcache.enabled' is removed. There is no way set combined mode = false. Means make BC as victim handler for LRU cache.
      This will be one more noticeable change when one uses BucketCache in File mode. Then the system table's data block(Including the META table) will be cached in Bucket Cache files only. Plain scan from META files alone test reveal that the throughput of file mode BC is almost half only. But for META entries we have RegionLocation cache at client side connections. So this would not be a big concern in a real cluster usage. Will check more on this and probably fix even when we do tiered BucketCache.
      Show
      Removed cacheDataInL1 option for HCD BucketCache is no longer the L2 for LRU on heap cache. When BC is used, data blocks will be strictly on BC only where as index/bloom blocks are on LRU L1 cache. Config 'hbase.bucketcache.combinedcache.enabled' is removed. There is no way set combined mode = false. Means make BC as victim handler for LRU cache. This will be one more noticeable change when one uses BucketCache in File mode. Then the system table's data block(Including the META table) will be cached in Bucket Cache files only. Plain scan from META files alone test reveal that the throughput of file mode BC is almost half only. But for META entries we have RegionLocation cache at client side connections. So this would not be a big concern in a real cluster usage. Will check more on this and probably fix even when we do tiered BucketCache.

    Description

      When Bucket cache is used, by default we dont configure it as an L2 cache alone. The default setting is combined mode ON where the data blocks to Bucket cache and index/bloom blocks go to LRU cache. But there is a way to turn this off and make LRU as L1 and Bucket cache as a victim handler for L1. It will be just L2.
      After the off heap read path optimization Bucket cache is no longer slower compared to L1. We have test results on data sizes from 12 GB. The Alibaba use case was also with 12 GB and they have observed a ~30% QPS improve over the LRU cache.
      This issue is to remove the option for combined mode = false. So when Bucket cache is in use, data blocks will go to it only and LRU will get only index /meta/bloom blocks. Bucket cache will no longer be configured as a victim handler for LRU.

      Note : WHen external cache is in use, there only the L1 L2 thing comes. LRU will be L1 and external cache act as its L2. That make full sense.

      Attachments

        1. HBASE-19357.patch
          52 kB
          Anoop Sam John
        2. HBASE-19357.patch
          52 kB
          Michael Stack
        3. HBASE-19357_V3.patch
          66 kB
          Anoop Sam John
        4. HBASE-19357_V3.patch
          66 kB
          Anoop Sam John
        5. HBASE-19357_V2.patch
          57 kB
          Anoop Sam John

        Issue Links

          causes

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-24317 Remove deprecated setCacheDataInL1 in HColumnDescriptor

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          New Feature - A new feature of the product, which has yet to be developed. HBASE-18300 Implement a Multi TieredBucketCache

          • Major - Major loss of function.
          • Open

          Activity

            People

              anoop.hbase Anoop Sam John
              anoop.hbase Anoop Sam John
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: