Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
0.21
-
None
-
None
-
RHEL 5.3, JDK 1.7.0_14
Description
When running a system test suite org.apache.qpid.systest.rest.BasicAuthRestTest with JDK 1.7 a deadlock might occur on shutting down the broker whilst it still tries to send some data over HTTPS:
Found one Java-level deadlock: ============================= "qtp538763660-62": waiting to lock monitor 0x000000000c91c440 (object 0x00000000f9404e28, a sun.security.ssl.SSLSocketImpl), which is held by "Broker-Configuration-Thread" "Broker-Configuration-Thread": waiting for ownable synchronizer 0x00000000f95c6278, (a java.util.concurrent.locks.ReentrantLock$NonfairSync), which is held by "qtp538763660-62" Java stack information for the threads listed above: =================================================== "qtp538763660-62": at sun.security.ssl.SSLSocketImpl.getConnectionState(SSLSocketImpl.java:649) - waiting to lock <0x00000000f9404e28> (a sun.security.ssl.SSLSocketImpl) at sun.security.ssl.SSLSocketImpl.isClosed(SSLSocketImpl.java:1446) at java.net.Socket.getTcpNoDelay(Socket.java:953) at sun.security.ssl.BaseSSLSocketImpl.getTcpNoDelay(BaseSSLSocketImpl.java:345) at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:819) at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:801) at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122) - locked <0x00000000f95c60b8> (a sun.security.ssl.AppOutputStream) at org.eclipse.jetty.io.ByteArrayBuffer.writeTo(ByteArrayBuffer.java:354) at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:159) at org.eclipse.jetty.io.bio.StreamEndPoint.flush(StreamEndPoint.java:189) at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:876) at org.eclipse.jetty.http.AbstractGenerator.flush(AbstractGenerator.java:438) at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:94) at org.eclipse.jetty.server.AbstractHttpConnection$Output.flush(AbstractHttpConnection.java:1016) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:173) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:101) at org.eclipse.jetty.util.ByteArrayISO8859Writer.writeTo(ByteArrayISO8859Writer.java:102) at org.eclipse.jetty.server.Response.sendError(Response.java:374) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:562) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) at org.eclipse.jetty.server.Server.handle(Server.java:348) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:452) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:884) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:938) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:630) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254) at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538) at java.lang.Thread.run(Thread.java:722) "Broker-Configuration-Thread": at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000f95c6278> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:214) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:290) at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:799) at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:672) at sun.security.ssl.SSLSocketImpl.sendAlert(SSLSocketImpl.java:2007) at sun.security.ssl.SSLSocketImpl.warning(SSLSocketImpl.java:1834) at sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1602) - locked <0x00000000f9404e28> (a sun.security.ssl.SSLSocketImpl) at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1538) at org.eclipse.jetty.io.bio.SocketEndPoint.close(SocketEndPoint.java:156) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.close(SocketConnector.java:233) at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.close(SslSocketConnector.java:613) at org.eclipse.jetty.server.bio.SocketConnector.doStop(SocketConnector.java:167) at org.eclipse.jetty.server.ssl.SslSocketConnector.doStop(SslSocketConnector.java:371) at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:84) - locked <0x00000000f94069a0> (a java.lang.Object) at org.eclipse.jetty.server.Server.doStop(Server.java:318) at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:84) - locked <0x00000000f9406dc0> (a java.lang.Object) at org.apache.qpid.server.management.plugin.HttpManagement.stop(HttpManagement.java:187) at org.apache.qpid.server.management.plugin.HttpManagement.setState(HttpManagement.java:156) at org.apache.qpid.server.model.adapter.AbstractAdapter.setDesiredState(AbstractAdapter.java:88) at org.apache.qpid.server.model.adapter.BrokerAdapter.changeState(BrokerAdapter.java:776) - locked <0x00000000f958c040> (a java.util.HashMap) at org.apache.qpid.server.model.adapter.BrokerAdapter.setState(BrokerAdapter.java:749) at org.apache.qpid.server.model.adapter.AbstractAdapter.setDesiredState(AbstractAdapter.java:88) at org.apache.qpid.server.configuration.updater.ChangeStateTask.call(ChangeStateTask.java:59) at org.apache.qpid.server.configuration.updater.ChangeStateTask.call(ChangeStateTask.java:28) at org.apache.qpid.server.configuration.updater.TaskExecutor.executeTask(TaskExecutor.java:213) at org.apache.qpid.server.configuration.updater.TaskExecutor.access$200(TaskExecutor.java:46) at org.apache.qpid.server.configuration.updater.TaskExecutor$CallableWrapper$1.run(TaskExecutor.java:252) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.qpid.server.configuration.updater.TaskExecutor$CallableWrapper.call(TaskExecutor.java:247) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) Found 1 deadlock.
It seems that the issue does not exist on JDK 1.6. I've been running tests for 2 days with JDK 1.6.0_33 but was not able to reproduce it.
Attachments
Attachments
Issue Links
- relates to
-
QPID-5294 [AMQP 1.0 JMS] Thread deadlock due to JVM bug JDK-8022788
- Closed