Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-3988

org.apache.zookeeper.server.NettyServerCnxn.receiveMessage throws NullPointerException

    XMLWordPrintableJSON

Details

    Description

      While upgrading K8S cluster, container running zookeeper will rollover one by one.
      During this rollover, Null Pointer Exception was observed as below.

      INFO  [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):Follower@292] - shutdown FollowerINFO  [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):Follower@292] - shutdown FollowerINFO  [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):LearnerZooKeeperServer@160] - Shutting downINFO  [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):ZooKeeperServer@784] - shutting downINFO  [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):RequestThrottler@244] - Shutting downINFO  [RequestThrottler:RequestThrottler@205] - Draining request throttler queueINFO  [RequestThrottler:RequestThrottler@181] - RequestThrottler shutdown. Dropped 0 requestsINFO  [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):FollowerRequestProcessor@148] - Shutting downINFO  [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):CommitProcessor@617] - Shutting downINFO  [FollowerRequestProcessor:1:FollowerRequestProcessor@112] - FollowerRequestProcessor exited loop!INFO  [CommitProcessor:1:CommitProcessor@406] - CommitProcessor exited loop!INFO  [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=0.0.0.0:2281):FinalRequestProcessor@662] - shutdown of request processor completeERROR [nioEventLoopGroup-4-22:NettyServerCnxnFactory$CnxnChannelHandler@329] - Unexpected exception in receivejava.lang.NullPointerException: null at org.apache.zookeeper.server.NettyServerCnxn.receiveMessage(NettyServerCnxn.java:515) ~[zookeeper-3.6.1.jar:3.6.1] at org.apache.zookeeper.server.NettyServerCnxn.processMessage(NettyServerCnxn.java:365) ~[zookeeper-3.6.1.jar:3.6.1] at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelRead(NettyServerCnxnFactory.java:326) [zookeeper-3.6.1.jar:3.6.1] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.48.Final.jar:4.1.48.Final] at java.lang.Thread.run(Thread.java:834) [?:?]WARN  [nioEventLoopGroup-4-22:NettyServerCnxnFactory$CnxnChannelHandler@273] - Exception caughtjava.lang.NullPointerException: null at org.apache.zookeeper.server.NettyServerCnxn.receiveMessage(NettyServerCnxn.java:515) ~[zookeeper-3.6.1.jar:3.6.1] at org.apache.zookeeper.server.NettyServerCnxn.processMessage(NettyServerCnxn.java:365) ~[zookeeper-3.6.1.jar:3.6.1] at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelRead(NettyServerCnxnFactory.java:326) ~[zookeeper-3.6.1.jar:3.6.1] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.48.Final.jar:4.1.48.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.48.Final.jar:4.1.48.Final] at java.lang.Thread.run(Thread.java:834) [?:?] 

      Expectation: This scenario should be handled and application should not throw such Null Pointer exception.

      Attachments

        Issue Links

          Activity

            People

              eolivelli Enrico Olivelli
              pratikthacker Pratik Thacker
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 5h
                  5h