Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-4089

blockCache contents report

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.99.0
    • None
    • None
    • Reviewed
    • Hide
      Show more detail on blockcache deployed in new BlockCache section under Tasks and above Regions; shows blockcache type, what for L1 and L2, help pointers, type-specific and L1/L2 stats as well as dumps of bucket/slab cache stats when bucket/slabs enabled. Keeps block cache age (works in Bucket and LruBlockCache, not for Slab). Adds JSON page with details on block caches and a facility for listing block cache content by file (useful finding hot files).
      Show
      Show more detail on blockcache deployed in new BlockCache section under Tasks and above Regions; shows blockcache type, what for L1 and L2, help pointers, type-specific and L1/L2 stats as well as dumps of bucket/slab cache stats when bucket/slabs enabled. Keeps block cache age (works in Bucket and LruBlockCache, not for Slab). Adds JSON page with details on block caches and a facility for listing block cache content by file (useful finding hot files).

    Description

      Summarized block-cache report for a RegionServer would be helpful. For example ...

      table1
      cf1 100 blocks, totalBytes=yyyyy, averageTimeInCache=XXXX hours
      cf2 200 blocks, totalBytes=zzzzz, averageTimeInCache=XXXX hours

      table2
      cf1 75 blocks, totalBytes=yyyyy, averageTimeInCache=XXXX hours
      cf2 150 blocks, totalBytes=zzzzz, averageTimeInCache=XXXX hours

      ... Etc.

      The current metrics list blockCacheSize and blockCacheFree, but there is no way to know what's in there. Any single block isn't really important, but the patterns of what CF/Table they came from, how big are they, and how long (on average) they've been in the cache, are important.

      No such interface exists in HRegionInterface. But I think it would be helpful from an operational perspective.

      Updated (7-29): Removing suggestion for UI. I would be happy just to get this report on a configured interval dumped to a log file.

      Attachments

        1. hbase_4089_blockcachereport.pdf
          145 kB
          Doug Meil
        2. java_blockcache_checkpoint_2011_08_11.patch
          13 kB
          Doug Meil
        3. 4089.wip.txt
          89 kB
          Michael Stack
        4. bc_l2_buckets.png
          140 kB
          Michael Stack
        5. bc_slabs.png
          110 kB
          Michael Stack
        6. bc_l1.png
          89 kB
          Michael Stack
        7. bc_stats.png
          109 kB
          Michael Stack
        8. bc_config.png
          121 kB
          Michael Stack
        9. bc_basic.png
          128 kB
          Michael Stack
        10. 4089.wip.v2.txt
          119 kB
          Michael Stack
        11. bc_slabs.png
          199 kB
          Michael Stack
        12. bc_buckets.png
          91 kB
          Michael Stack
        13. bc_showing_buckets_start.png
          50 kB
          Michael Stack
        14. bc_bucketcache.png
          98 kB
          Michael Stack
        15. 4089v3.txt
          147 kB
          Michael Stack
        16. 4089v4.txt
          149 kB
          Michael Stack
        17. 4089v6.txt
          151 kB
          Michael Stack
        18. 4089v7.txt
          151 kB
          Michael Stack
        19. 4089v7.txt
          151 kB
          Michael Stack
        20. 4089v8.txt
          151 kB
          Michael Stack
        There are no Sub-Tasks for this issue.

        Activity

          People

            stack Michael Stack
            dmeil Doug Meil
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: