Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-14048 The Next Generation of the Consumer Rebalance Protocol
  3. KAFKA-15276

Implement event plumbing for ConsumerRebalanceListener callbacks

    XMLWordPrintableJSON

Details

    Description

      Provide the Java client support for the consumer group partition assignment logic, including:

      • Calculate the difference between the current partition assignment and that returned in the ConsumerGroupHeartbeatResponse RPC response
      • Ensure we handle the case where changes to the assignment take multiple passes of RequestManager.poll()
      • Integrate the mechanism to invoke the user’s rebalance callback

      This task is part of the work to implement support for the new KIP-848 consumer group protocol.

      One challenge to solve here is that callback execution, when triggered from the unsubscribe, would require the user to poll for complete the callbacks and then return in the unsubscribe (legacy unubscribe does not require user poll). We might need to re-think callbacks execution considering that they do need to be executed in the app thread, but should not require the user to poll (seeing that they are not events directly generated by the user. They are generated internally by the background thread, to be executed in the app thread) 

      Attachments

        Issue Links

          Activity

            People

              kirktrue Kirk True
              kirktrue Kirk True
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: