Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-7956

"nodetool compactionhistory" crashes because of low heap size (GC overhead limit exceeded)

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Low
    • Resolution: Fixed
    • 2.0.11, 2.1.1
    • None
    • Cassandra 2.0.8

    • Low

    Description

      ]# nodetool compactionhistory
      Compaction History:
      Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
              at java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:967)
              at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1782)
              at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
              at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
              at java.util.HashMap.readObject(HashMap.java:1180)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
              at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
              at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
              at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
              at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
              at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:500)
              at javax.management.openmbean.TabularDataSupport.readObject(TabularDataSupport.java:912)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
              at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
              at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
              at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
              at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
              at sun.rmi.server.UnicastRef.unmarshalValue(UnicastRef.java:325)
              at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:174)
              at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
              at javax.management.remote.rmi.RMIConnectionImpl_Stub.getAttribute(Unknown Source)
              at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(RMIConnector.java:906)
              at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:267)
              at com.sun.proxy.$Proxy3.getCompactionHistory(Unknown Source)
      

      nodetool starts with -Xmx32m. This seems to be not enough at least in my case to show the history. I am not sure what would the appropriate amount be but increasing it to 128m definitely solves the problem. Output from modified nodetool attached.

      Attachments

        1. 7956.txt
          0.4 kB
          Michael Shuler
        2. nodetool_compactionhistory_128m_heap_output.txt.gz
          1014 kB
          Nikolai Grigoriev

        Issue Links

          Activity

            People

              mshuler Michael Shuler
              ngrigoriev@gmail.com Nikolai Grigoriev
              Michael Shuler
              Jonathan Ellis
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: