Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Auto Closed
-
0.8.2.0
-
None
-
None
Description
Short version: Like many other tools, mirror-maker's consumer defaults to using the localhost zookeeper instance when no specific zookeeper source is specified. It shouldn't do this. MM should also have a sanity check that the source and destination clusters are different.
Long version: We run multiple clusters, all using mirrormaker to replicate to the master cluster. The kafka, zookeeper, and mirrormaker instances all run on the same nodes in the master cluster since the hardware can more than handle the load. We were doing some zookeeper maintenance on one of our remote clusters recently which accidentally caused our configuration manager (chef) to generate empty zkConnect strings for some mirrormaker instances. These instances defaulted to localhost and started mirroring from the master cluster back to itself, an infinite replication loop that caused all sorts of havok.
We were able to recover gracefully and we've added additional safe-guards on our end, but mirror-maker is at least partially at fault here as well. There is no reason for it to treat an empty string as anything but an error - especially not localhost, which is typically the target cluster, not the source. Additionally, it should be trivial and very useful for mirrormaker to verify it is not consuming and producing from the same cluster; I can think of no legitimate use case for this kind of cycle.
If you need any clarification or additional information, please let me know.