Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4633

Always use regex pattern subscription to avoid auto create topics

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.10.2.0
    • streams

    Description

      In KafkaConsumer, a metadata update is requested whenever subscribe(List<String> topics ..) is called. And when such a metadata request is sent to the broker upon the first poll call, it will cause the broker to auto-create any topics that do not exist if the broker-side config topic.auto.create is turned on.

      In order to work around this issue until the config is default to false and gradually be deprecated, we will let Streams to always use the other subscribe function with regex pattern, which will send the metadata request with empty topic list and hence won't trigger broker-side auto topic creation.

      The side-effect is that the metadata response will be larger, since it contains all the topic infos; but since we only refresh it infrequently this will add negligible overhead.

      Attachments

        Issue Links

          Activity

            People

              guozhang Guozhang Wang
              guozhang Guozhang Wang
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: