[CAMEL-17051] camel-kafka: Need the ability to pause kafka consumer - ASF JIRA

XML

Word

Printable

JSON

Details

Type: New Feature
Status: Resolved
Priority: Minor
Resolution: Fixed
Affects Version/s: 3.12.0
Fix Version/s: 3.17.0
Component/s: camel-kafka
Labels:
None

Estimated Complexity:
Unknown

Description

When handling transient errors that have exhausted retry limits, sometimes pausing the Kafka consumer is the best way to handle an outage. A great example of this would be the Facebook outage that happen this week.

One might suggest that stopping the consumer work would for this use case. This would of course work but would require a manual restart. Also, you want to try to avoid a rebalance of the consumer.

I would like to see a new strategy be created that would allow for pause and resume to be configured.

Ideally, the resume could be configured based on a health check.

Example code: https://github.com/jeanlouisboudart/retriable-consumer/blob/master/consumer/src/main/java/com/sample/InfiniteRetriesConsumer.java