Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.8.0
-
None
-
None
-
None
Description
The OOME errors seem to happen after executing these tests:
org.apache.geode.internal.cache.wan.offheap.SerialGatewaySenderQueueOffHeapDUnitTest > testCreateMaximumSenders
testCreateMaximumPlusOneSenders
REPRODUCED:
DistributedTestOpenJDK11 job 105 failed in the same way.
http://files.apachegeode-ci.info/builds/apache-develop-main/1.9.0-build.140/test-results/distributedTest/1542056138/
Log (abridged)
org.apache.geode.test.dunit.RMIException: While invoking org.apache.geode.internal.cache.wan.serial.SerialGatewaySenderQueueDUnitTest$$Lambda$377/0x0000000840571840.run in VM 2 running on Host 72ac34082783 with 8 VMs at org.apache.geode.test.dunit.VM.invoke(VM.java:433) at org.apache.geode.test.dunit.VM.invoke(VM.java:402) at org.apache.geode.test.dunit.VM.invoke(VM.java:361) at org.apache.geode.internal.cache.wan.serial.SerialGatewaySenderQueueDUnitTest.testCreateMaximumSenders(SerialGatewaySenderQueueDUnitTest.java:370) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:566) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:117) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:566) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:155) at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:137) at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63) at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55) at java.lang.Thread.run(Thread.java:834) Caused by: org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in org.apache.geode.internal.cache.wan.WANTestBase that uses long was not fulfilled within 300 seconds. at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:145) at org.awaitility.core.CallableCondition.await(CallableCondition.java:79) at org.awaitility.core.CallableCondition.await(CallableCondition.java:27) at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:890) at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:848) at org.apache.geode.internal.cache.wan.WANTestBase.verifyListenerEvents(WANTestBase.java:3540) at org.apache.geode.internal.cache.wan.serial.SerialGatewaySenderQueueDUnitTest.lambda$testCreateMaximumSenders$faf964a3$1(SerialGatewaySenderQueueDUnitTest.java:370) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:566) at hydra.MethExecutor.executeObject(MethExecutor.java:244) at org.apache.geode.test.dunit.standalone.RemoteDUnitVM.executeMethodOnObject(RemoteDUnitVM.java:70) at jdk.internal.reflect.GeneratedMethodAccessor1.invoke(Unknown Source) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:566) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:359) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:562) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:796) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:677) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:676) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ... 1 more java.lang.AssertionError: Suspicious strings were written to the log during this run. Fix the strings or use IgnoredException.addIgnoredException to ignore. ----------------------------------------------------------------------- Found suspect string in log4j at line 6124 [fatal 2018/11/12 20:06:18.048 UTC <Geode Membership View Creator> tid=280] Possible loss of quorum due to the loss of 1 cache processes: [172.17.0.23(483)<v1>:41004] ----------------------------------------------------------------------- Found suspect string in log4j at line 6126 [fatal 2018/11/12 20:06:19.050 UTC <Geode Membership View Creator> tid=280] Membership service failure: Exiting due to possible network partition event due to loss of 1 cache processes: [172.17.0.23(483)<v1>:41004] org.apache.geode.ForcedDisconnectException: Exiting due to possible network partition event due to loss of 1 cache processes: [172.17.0.23(483)<v1>:41004] at org.apache.geode.distributed.internal.membership.gms.mgr.GMSMembershipManager.forceDisconnect(GMSMembershipManager.java:2503) at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.forceDisconnect(GMSJoinLeave.java:1049) at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.access$1500(GMSJoinLeave.java:90) at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.prepareAndSendView(GMSJoinLeave.java:2506) at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.createAndSendView(GMSJoinLeave.java:2483) at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.run(GMSJoinLeave.java:2267) ----------------------------------------------------------------------- Found suspect string in log4j at line 6749 [fatal 2018/11/12 20:10:30.023 UTC <Asynchronous disk writer for region ln-59> tid=2640] Fatal error from asynchronous flusher thread java.lang.OutOfMemoryError: Java heap space ----------------------------------------------------------------------- Found suspect string in log4j at line 6752 [fatal 2018/11/12 20:10:30.023 UTC <Asynchronous disk writer for region ln-67> tid=2707] Fatal error from asynchronous flusher thread java.lang.OutOfMemoryError: Java heap space ----------------------------------------------------------------------- Found suspect string in log4j at line 6755 [fatal 2018/11/12 20:10:30.024 UTC <Asynchronous disk writer for region ln-90> tid=2898] Fatal error from asynchronous flusher thread java.lang.OutOfMemoryError: Java heap space ----------------------------------------------------------------------- Found suspect string in log4j at line 6758 [fatal 2018/11/12 20:10:30.024 UTC <Asynchronous disk writer for region ln-81> tid=2824] Fatal error from asynchronous flusher thread java.lang.OutOfMemoryError: Java heap space ----------------------------------------------------------------------- Found suspect string in log4j at line 6761 [error 2018/11/12 20:10:30.024 UTC <unicast receiver,72ac34082783-15304> tid=2012] JGRP000190: failed receiving packet java.lang.OutOfMemoryError: Java heap space ----------------------------------------------------------------------- Found suspect string in log4j at line 6764 [fatal 2018/11/12 20:10:30.024 UTC <Asynchronous disk writer for region ln-109> tid=3052] Fatal error from asynchronous flusher thread java.lang.OutOfMemoryError: Java heap space ----------------------------------------------------------------------- Found suspect string in log4j at line 6843
Attachments
Issue Links
- is a clone of
-
GEODE-5826 java.lang.OutOfMemoryError: Java heap space when running DUnit tests using java10
- Resolved