Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8858

Kafka Streams - Failed to Rebalance Error and stream consumer stuck for some reason

    XMLWordPrintableJSON

Details

    Description

      I have a basic Kafka Streams application that reads from a topic, performs a rolling aggregate, and performs a join to publish to an agg_topic. Our project has the timeout failure in Kafka 2.1.1 env and we don't know the reason yet.

      Our stream consumer stuck for some reason. 

      After we changed our group id to another one it became normal. So seems offset data for this consumer is corrupted.

      Can you help us please to resolve this problem to be able to revert us to the previous consumer name because we have many inconveniences due to this.

      Ping me pls if you will need some additional info.

      Our temporary workaround is to disable the exactly_once config which skips the initializing transactional state. Also offset reseted for corrupted partition, with no effect.

      Full problem description in log:

      
      

      [2019-08-30 14:20:02.168] [abc-streamer-StreamThread-21] ERROR org.apache.kafka.streams.processor.internals.StreamThread:273 - stream-thread [abc-streamer-StreamThread-21] Error caught during partition assignment, will abort the current process and re-throw at the end of rebalance: {}
      org.apache.kafka.common.errors.TimeoutException: Timeout expired while initializing transactional state in 60000ms.
      [2019-08-30 14:21:35.407] [abc-streamer-StreamThread-14] ERROR org.apache.kafka.streams.processor.internals.StreamThread:273 - stream-thread [abc-streamer-StreamThread-14] Error caught during partition assignment, will abort the current process and re-throw at the end of rebalance: {}
      org.apache.kafka.common.errors.TimeoutException: Timeout expired while initializing transactional state in 60000ms.
      [2019-08-30 14:22:58.487] [abc-streamer-StreamThread-13] ERROR org.apache.kafka.streams.processor.internals.StreamThread:273 - stream-thread [abc-streamer-StreamThread-13] Error caught during partition assignment, will abort the current process and re-throw at the end of rebalance: {}
      org.apache.kafka.common.errors.TimeoutException: Timeout expired while initializing transactional state in 60000ms.

      
      

       

       

       

      Attachments

        Issue Links

          Activity

            People

              guozhang Guozhang Wang
              baronic Ante B.
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: