Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-15309

Add custom error handler to Producer

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • producer

    Description

      The producer collects multiple records into batches, and a single record specific error might fail the whole batch (eg, `RecordTooLargeException`).

      This ticket suggests to add a per-record error handler, that allows user to opt into skipping bad records without failing the whole batch (similar to Kafka Streams `ProductionExceptionHandler`).

      The fix of https://issues.apache.org/jira/browse/KAFKA-9279 caused https://issues.apache.org/jira/browse/KAFKA-15259 which inspired this ticket.

      Another example for which a production exception handler could be useful, if a user tries to write into a non-existing topic, which returns a retryable error code; with infinite retries the producer would hang retrying forever. A handler could help to break the infinite retry loop.

      Attachments

        1. KafkaProducerReproducer.java
          5 kB
          Tomonari Yamashita
        2. app.log
          632 kB
          Tomonari Yamashita

        Issue Links

          Activity

            People

              alisa23 Alieh Saeedi
              mjsax Matthias J. Sax
              Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: