Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-28059

Use correct units in RegionLoad#getStoreUncompressedSizeMB()

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.5.5
    • 2.6.0, 2.4.18, 2.5.6
    • Admin
    • None

    Description

      When I run a snippet of code like this:

      Map<byte[], RegionLoad> regionLoadMap = admin
        .getClusterStatus()
        .getLoad(
          ServerName.parseServerName(
            "my-server.my-company.net,60020,1693513660506"
          )
        )
        .getRegionsLoad();
      for (byte[] startKey : regionLoadMap.keySet()) {
        RegionLoad regionLoad = regionLoadMap.get(startKey);
        LOG.info("Region {}: {}", Bytes.toStringBinary(startKey), regionLoad);
      } 

      I get logs like this:

      Region <redacted-table-name>,<redacted row key>,1659484033280.2b89407a1223720344bed425bf3c29b0.: numberOfStores=1, numberOfStorefiles=3, storeRefCount=0, storefileUncompressedSizeMB=17998848, lastMajorCompactionTimestamp=1693211464712, storefileSizeMB=5895, compressionRatio=0.0003, memstoreSizeMB=1, readRequestsCount=118899553, writeRequestsCount=731192, rootIndexSizeKB=9, totalStaticIndexSizeKB=10413, totalStaticBloomSizeKB=6592, totalCompactingKVs=0, currentCompactedKVs=0, compactionProgressPct=NaN, completeSequenceId=78093096, dataLocality=1.0

      The storefileUncompressedSizeMB is vastly larger than the storefileSizeMB, so much more that it's not believable. I checked the store files in question in this instance. Adding up the uncompressed size reported in the HFile trailers sums to 5895 MiB, exactly 1024 times less than the reported 17998848.

      The reason for the misreporting is that RegionLoad#getStoreUncompressedSizeMB() converts the value from a Size object wrong.

       

      Attachments

        Activity

          People

            charlesconnell Charles Connell
            charlesconnell Charles Connell
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: