Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1355

Reduce/optimize update metadata requests sent during leader election

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.8.1
    • 0.8.1.1
    • None
    • None

    Description

      This is part of the investigation into slow shutdowns in 0.8.1. While
      logging contributes to bulk of the regression, this one also adds
      quite a bit of overhead:

      In addLeaderAndIsrRequest (called for every partition that is led by the
      broker being shut down) we also add an UpdateMetadataRequest - each call to
      addUpdateMetadataRequests does two traversals over all (global)
      partitions. I think it should be straightforward to optimize this a bit.

      Marking as critical, since it is not as big an overhead as the logging.

      Attachments

        1. KAFKA-1355.patch
          16 kB
          Joel Jacob Koshy
        2. KAFKA-1355_2014-04-17_14:48:57.patch
          16 kB
          Joel Jacob Koshy
        3. KAFKA-1355_2014-04-04_13:51:22.patch
          16 kB
          Joel Jacob Koshy
        4. KAFKA-1355_2014-04-04_13:48:34.patch
          16 kB
          Joel Jacob Koshy

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jjkoshy Joel Jacob Koshy
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: