Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-4536

Zookeeper quorum formation fails when TLS is enabled in k8s env

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Blocker
    • Resolution: Unresolved
    • 3.7.0
    • None
    • leaderElection, quorum
    • None
    • Kubernetes 1.21.1

    Description

      We have three(3) node zookeeper cluster running as a pod on Kubernetes cluster, zookeeper quorum formation fails with TLS handshake error, as the server name in the https request does not match with any of the SANs in the certificate configured for zookeeper server. Server name in the request is of the form "x-x-x-x.kubernetes.default.svc.cluster.local" (where x-x-x-x is the IP address of the POD), and I am unable to understand the reason behind pre-pending FQDN with a IP address.

       

      Please find below the extract of the error logs from the zookeeper POD

       

      2022-04-12T12:48:03.551+0200 [myid:] - ERROR [ListenerHandler-0.0.0.0/0.0.0.0:3888:ZKTrustManager@161] - Failed to verify host address: 192.168.140.200
      javax.net.ssl.SSLPeerUnverifiedException: Certificate for <192.168.140.200> doesn't match any of the subject alternative names: [eric-data-coordinator-zk, eric-data-coordinator-zk.eda-esmalir, eric-data-coordinator-zk.eda-esmalir.svc, eric-data-coordinator-zk.eda-esmalir.svc.cluster.local, *.eric-data-coordinator-zk-ensemble-service.eda-esmalir.svc.cluster.local, certified-scrape-target]
      at org.apache.zookeeper.common.ZKHostnameVerifier.matchIPAddress(ZKHostnameVerifier.java:197) ~[zookeeper-3.7.0.jar:3.7.0]
      at org.apache.zookeeper.common.ZKHostnameVerifier.verify(ZKHostnameVerifier.java:165) ~[zookeeper-3.7.0.jar:3.7.0]
      at org.apache.zookeeper.common.ZKTrustManager.performHostVerification(ZKTrustManager.java:151) [zookeeper-3.7.0.jar:3.7.0]
      at org.apache.zookeeper.common.ZKTrustManager.checkClientTrusted(ZKTrustManager.java:79) [zookeeper-3.7.0.jar:3.7.0]
      at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkClientCerts(CertificateMessage.java:688) [?:?]
      at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:411) [?:?]
      at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:375) [?:?]
      at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392) [?:?]
      at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443) [?:?]
      at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421) [?:?]
      at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182) [?:?]
      at sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) [?:?]
      at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1426) [?:?]
      at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1336) [?:?]
      at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450) [?:?]
      at sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:841) [?:?]
      at sun.security.ssl.SSLSocketImpl.getSession(SSLSocketImpl.java:366) [?:?]
      at org.apache.zookeeper.server.quorum.UnifiedServerSocket$UnifiedSocket.detectMode(UnifiedServerSocket.java:269) [zookeeper-3.7.0.jar:3.7.0]
      at org.apache.zookeeper.server.quorum.UnifiedServerSocket$UnifiedSocket.getSocket(UnifiedServerSocket.java:298) [zookeeper-3.7.0.jar:3.7.0]
      at org.apache.zookeeper.server.quorum.UnifiedServerSocket$UnifiedSocket.access$400(UnifiedServerSocket.java:172) [zookeeper-3.7.0.jar:3.7.0]
      at org.apache.zookeeper.server.quorum.UnifiedServerSocket$UnifiedInputStream.getRealInputStream(UnifiedServerSocket.java:699) [zookeeper-3.7.0.jar:3.7.0]
      at org.apache.zookeeper.server.quorum.UnifiedServerSocket$UnifiedInputStream.read(UnifiedServerSocket.java:693) [zookeeper-3.7.0.jar:3.7.0]
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:252) [?:?]
      at java.io.BufferedInputStream.read1(BufferedInputStream.java:292) [?:?]
      at java.io.BufferedInputStream.read(BufferedInputStream.java:351) [?:?]
      at java.io.DataInputStream.readFully(DataInputStream.java:200) [?:?]
      at java.io.DataInputStream.readLong(DataInputStream.java:421) [?:?]
      at org.apache.zookeeper.server.quorum.QuorumCnxManager.handleConnection(QuorumCnxManager.java:602) [zookeeper-3.7.0.jar:3.7.0]
      at org.apache.zookeeper.server.quorum.QuorumCnxManager.receiveConnection(QuorumCnxManager.java:555) [zookeeper-3.7.0.jar:3.7.0]
      at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener$ListenerHandler.acceptConnections(QuorumCnxManager.java:1080) [zookeeper-3.7.0.jar:3.7.0]
      at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener$ListenerHandler.run(QuorumCnxManager.java:1034) [zookeeper-3.7.0.jar:3.7.0]
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
      at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
      at java.lang.Thread.run(Thread.java:829) [?:?]
       
      

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            Sai kiran Sai kiran
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: