Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
3.6.1
-
K8S
Description
While upgrading K8S cluster, container running zookeeper will rollover one by one.
During this rollover, Null Pointer Exception was observed as below.
INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):Follower@292] - shutdown FollowerINFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):Follower@292] - shutdown FollowerINFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):LearnerZooKeeperServer@160] - Shutting downINFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):ZooKeeperServer@784] - shutting downINFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):RequestThrottler@244] - Shutting downINFO [RequestThrottler:RequestThrottler@205] - Draining request throttler queueINFO [RequestThrottler:RequestThrottler@181] - RequestThrottler shutdown. Dropped 0 requestsINFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):FollowerRequestProcessor@148] - Shutting downINFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):CommitProcessor@617] - Shutting downINFO [FollowerRequestProcessor:1:FollowerRequestProcessor@112] - FollowerRequestProcessor exited loop!INFO [CommitProcessor:1:CommitProcessor@406] - CommitProcessor exited loop!INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):FinalRequestProcessor@662] - shutdown of request processor completeERROR [nioEventLoopGroup-4-22:NettyServerCnxnFactory$CnxnChannelHandler@329] - Unexpected exception in receivejava.lang.NullPointerException: null at org.apache.zookeeper.server.NettyServerCnxn.receiveMessage(NettyServerCnxn.java:515) ~[zookeeper-3.6.1.jar:3.6.1] at org.apache.zookeeper.server.NettyServerCnxn.processMessage(NettyServerCnxn.java:365) ~[zookeeper-3.6.1.jar:3.6.1] at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelRead(NettyServerCnxnFactory.java:326) [zookeeper-3.6.1.jar:3.6.1] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.48.Final.jar:4.1.48.Final] at java.lang.Thread.run(Thread.java:834) [?:?]WARN [nioEventLoopGroup-4-22:NettyServerCnxnFactory$CnxnChannelHandler@273] - Exception caughtjava.lang.NullPointerException: null at org.apache.zookeeper.server.NettyServerCnxn.receiveMessage(NettyServerCnxn.java:515) ~[zookeeper-3.6.1.jar:3.6.1] at org.apache.zookeeper.server.NettyServerCnxn.processMessage(NettyServerCnxn.java:365) ~[zookeeper-3.6.1.jar:3.6.1] at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelRead(NettyServerCnxnFactory.java:326) ~[zookeeper-3.6.1.jar:3.6.1] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.48.Final.jar:4.1.48.Final] at java.lang.Thread.run(Thread.java:834) [?:?]
Expectation: This scenario should be handled and application should not throw such Null Pointer exception.
Attachments
Issue Links
- relates to
-
ZOOKEEPER-4453 NettyServerCnxnFactory: allow to configure the early TLS connection drop feature
- Resolved
-
ZOOKEEPER-3682 Stop initializing new SSL connection if ZK server is shutting down
- Closed
- links to
(1 links to)