Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-1846

Cached InetSocketAddresses prevent proper dynamic DNS resolution

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 3.4.6
    • None
    • quorum
    • Hide
      Forces the re-resolve, on error, of the Peers' Hostname to IP address, which is an issue in virtual/cloud environments where IPs are assigned dynamically upon every container startup.
      If the Hostname is unresolvable or the connection fails (IP change), this DNS refresh process is immediately triggered.
      Show
      Forces the re-resolve, on error, of the Peers' Hostname to IP address, which is an issue in virtual/cloud environments where IPs are assigned dynamically upon every container startup. If the Hostname is unresolvable or the connection fails (IP change), this DNS refresh process is immediately triggered.

    Description

      The class QuorumPeer maintains a Map<Long, QuorumServer> quorumPeers.
      Each QuorumServer is created with an instance of InetSocketAddress electionAddr, and holds it forever.

      I believe this is why the ZooKeeper servers can't resolve each other dynamically: If a ZooKeeper in the ensemble cannot be resolved at startup, it will never be resolved (until restart of the JVM), constantly failing with an UnknownHostException, even when the node is back up and reachable.

      I would suggest to recreate an InetSocketAddress every time we retry the connection.

      Attachments

        1. Learner.java
          22 kB
          Mark Duske
        2. QuorumPeer.java
          41 kB
          Mark Duske
        3. QuorumCnxManager.java
          31 kB
          Mark Duske
        4. DynamicIP.java.patch
          3 kB
          Mark Duske

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. ZOOKEEPER-1506 Re-try DNS hostname -> IP resolution if node connection fails

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          links to

          Web Link GitHub Pull Request #37

          Activity

            People

              Unassigned Unassigned
              benjamin.jaton Benjamin Jaton
              Votes:
              1 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m