Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4299

SpillableMemoryManager assumes tenured heap incorrectly

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.13.0
    • 0.14.0
    • None
    • None
    • Patch Available
    • Reviewed

    Description

      for (MemoryPoolMXBean b: mpbeans) {
                  log.debug("Found heap (" + b.getName() +
                      ") of type " + b.getType());
                  if (b.getType() == MemoryType.HEAP) {
                      /* Here we are making the leap of faith that the biggest
                       * heap is the tenured heap
                       */
                      long size = b.getUsage().getMax();
                      totalSize += size;
                      if (size > biggestSize) {
                          biggestSize = size;
                          biggestHeap = b;
                      }
                  }
              }
      

      A memory pool being the biggest MemoryType.HEAP does not guarantee it being tenured. Moreover, we must check whether usage threshold is supported by heap before trying to set usage threshold on it.

      Here is the stacktrace that resulted from this bug

      java.lang.UnsupportedOperationException: Usage threshold is not supported
      at sun.management.MemoryPoolImpl.setUsageThreshold(MemoryPoolImpl.java:114)
      at org.apache.pig.impl.util.SpillableMemoryManager.<init>(SpillableMemoryManager.java:130)
      at org.apache.pig.impl.util.SpillableMemoryManager.getInstance(SpillableMemoryManager.java:135)
      at org.apache.pig.data.BagFactory.<init>(BagFactory.java:123)
      at org.apache.pig.data.DefaultBagFactory.<init>(DefaultBagFactory.java:69)
      at org.apache.pig.data.BagFactory.getInstance(BagFactory.java:81)
      at search.dashboard.VariableLengthTupleToBag.<clinit>(VariableLengthTupleToBag.java:27)

      Attachments

        1. PIG-4299_1.patch
          3 kB
          Prashant Kommireddi

        Activity

          People

            prkommireddi Prashant Kommireddi
            prkommireddi Prashant Kommireddi
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: