Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15179 Cell/DBB end-to-end on the write-path
  3. HBASE-15180

Reduce garbage created while reading Cells from Codec Decoder

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.98.0
    • 2.0.0
    • regionserver
    • None
    • Reviewed

    Description

      In KeyValueDecoder#parseCell (Default Codec decoder) we use KeyValueUtil#iscreate to read cells from the InputStream. Here we 1st create a byte[] of length 4 and read the cell length and then an array of Cell's length and read in cell bytes into it and create a KV.
      Actually in server we read the reqs into a byte[] and CellScanner is created on top of a ByteArrayInputStream on top of this. By default in write path, we have MSLAB usage ON. So while adding Cells to memstore, we will copy the Cell bytes to MSLAB memory chunks (default 2 MB size) and recreate Cells over that bytes. So there is no issue if we create Cells over the RPC read byte[] directly here in Decoder. No need for 2 byte[] creation and copy for every Cell in request.

      My plan is to make a Cell aware ByteArrayInputStream which can read Cells directly from it.

      Same Codec path is used in client side also. There better we can avoid this direct Cell create and continue to do the copy to smaller byte[]s path. Plan to introduce some thing like a CodecContext associated with every Codec instance which can say the server/client context.

      Attachments

        1. HBASE-15180_V7.patch
          16 kB
          Anoop Sam John
        2. HBASE-15180_V6.patch
          15 kB
          Anoop Sam John
        3. HBASE-15180_V4.patch
          10 kB
          Anoop Sam John
        4. HBASE-15180_V2.patch
          20 kB
          Anoop Sam John
        5. HBASE-15180.patch
          21 kB
          Anoop Sam John

        Issue Links

          Activity

            People

              anoop.hbase Anoop Sam John
              anoop.hbase Anoop Sam John
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: