Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-227

Upgrade KafkaSystemProducer to new API

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.6.0
    • 0.9.0
    • kafka

    Description

      Kafka has upgraded their producer API in 0.8.1/0.8.2 to have a new Java-based API. It's much improved both in performance, and features. We should upgrade KafkaSystemProducer to use it.

      Initially, we should keep the old producer as well, and add a config to switch back and forth. Once we've run it for a while, and things look stable, we can deprecate and delete the old producer.

      Attachments

        1. rb29899.patch
          69 kB
          Navina Ramesh
        2. rb29899-1.patch
          68 kB
          Navina Ramesh

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-1794 Make config and config defaults accessible to clients

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-1863 Exception categories / hierarchy in clients

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SAMZA-459 Explicit flush for individual output streams

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. SAMZA-523 Remove custom implementation of MockKafkaProducer, once the missing features are added to MockProducer in Kafka

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. SAMZA-525 Update access to kafka's ProducerConfig variables

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-1865 Add a flush() call to the new producer API

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SAMZA-522 Upgrade to Kafka 0.8.2 instead of Kafka 0.8.2-beta

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SAMZA-524 Update KafkaSystemProducer after Kafka provides a flush behavior

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SAMZA-546 Add config option max.in.flight.requests.per.connection

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              navina Navina Ramesh
              criccomini Chris Riccomini
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: