Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21355

HStore's storeSize is calculated repeatedly which causing the confusing region split

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      When testing the branch-2's write performance in our internal cluster, we found that the region will be inexplicably split.

      We use the default ConstantSizeRegionSplitPolicy and hbase.hregion.max.filesize=40G,but the region will be split even if its bytes size is less than 40G(only ~6G).

      Checked the code, I found that the following path will accumulate the store's storeSize to a very big value, because the path has no reset..

      RsRpcServices#getRegionInfo
            -> HRegion#isMergeable
                 -> HRegion#hasReferences
                      -> HStore#hasReferences
                              -> HStore#openStoreFiles
      

      BTW, we seems forget to maintain the read replica's storeSize when refresh the store files.

      Some comment here, I move the storeSize calculation out of loadStoreFiles() method, because the secondary read replica's refreshStoreFiles() will also use loadStoreFiles() to refresh its store files and update the storeSize in the completeCompaction(..) in the final (just like compaction.) , so no need calculate the storeSize twice..

      Attachments

        1. HBASE-21355.v1.patch
          6 kB
          Zheng Hu
        2. HBASE-21355.branch-1.patch
          6 kB
          Zheng Hu
        3. HBASE-21355.addendum.patch
          2 kB
          Zheng Hu
        4. HBASE-21355.addendum.patch
          3 kB
          Zheng Hu

        Issue Links

          Activity

            People

              openinx Zheng Hu
              openinx Zheng Hu
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: