Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-12731

Heap occupancy based client pushback

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.0.0, 0.98.10, 1.1.0
    • None
    • None
    • Reviewed
    • Hide
      This feature incorporates reported regionserver heap occupancy in the (optional) client pushback calculations. If client pushback is enabled, the exponential backoff policy will take heap occupancy into account, should it exceed "hbase.heap.occupancy.low_water_mark" percentage of the heap (default 0.95). Once above the low water mark, heap occupancy is an additional factor scaling from 0.1 up to 1.0 at "hbase.heap.occupancy.high_water_mark" (default 0.98). At or above the high water mark the client will use the maximum configured backoff.
      Show
      This feature incorporates reported regionserver heap occupancy in the (optional) client pushback calculations. If client pushback is enabled, the exponential backoff policy will take heap occupancy into account, should it exceed "hbase.heap.occupancy.low_water_mark" percentage of the heap (default 0.95). Once above the low water mark, heap occupancy is an additional factor scaling from 0.1 up to 1.0 at "hbase.heap.occupancy.high_water_mark" (default 0.98). At or above the high water mark the client will use the maximum configured backoff.

    Description

      If the heap occupancy of a RegionServer is beyond a configurable high water mark (suggestions: 95%, 98%) then we should reject all user RPCs and only allow administrative RPCs until occupancy has dropped below a configurable low water mark (suggestions: 92%).

      Implement building on the HBASE-5162 changes.

      It might be expensive to check heap occupancy, in which case we can sample it periodically with a chore and use the last known value in pushback calculations.

      Attachments

        1. HBASE-12731.patch
          36 kB
          Andrew Kyle Purtell
        2. HBASE-12731.patch
          36 kB
          Andrew Kyle Purtell
        3. HBASE-12731.patch
          35 kB
          Andrew Kyle Purtell
        4. HBASE-12731.patch
          29 kB
          Andrew Kyle Purtell
        5. HBASE-12731-0.98.patch
          36 kB
          Andrew Kyle Purtell
        6. HBASE-12731-0.98.patch
          36 kB
          Andrew Kyle Purtell

        Issue Links

          relates to

          New Feature - A new feature of the product, which has yet to be developed. HBASE-5162 Basic client pushback mechanism

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-14756 Break out ClientBackoffPolicy factors into configurable and composable components

          • Major - Major loss of function.
          • Closed

          Umbrella - An overarching type made of sub-tasks HBASE-12702 Backport client pushback

          • Major - Major loss of function.
          • Closed

          Activity

            People

              apurtell Andrew Kyle Purtell
              apurtell Andrew Kyle Purtell
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: