Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5451

Kafka Connect should scan classpath asynchronously

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Won't Do
    • 0.11.0.0
    • None
    • connect

    Description

      When Kafka Connect workers start up, they scan the classpath and module paths for connectors, transformations, and converters. This takes anywhere from 15-30sec or longer depending upon how many JARs are included. Currently, this scanning is done synchronously during startup of the Kafka Connect workers, even though the workers may not need the result of the scan.

      The scanning logic should be asynchronous and should only block any components that require the result of the scan. This will improve startup time of the workers.

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. KAFKA-6208 Reduce startup time for Kafka Connect workers

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-5450 Scripts to startup Connect in system tests have too short a timeout

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. KAFKA-14627 Modernize Connect plugin discovery

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              rhauch Randall Hauch
              rhauch Randall Hauch
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 24h
                  24h
                  Remaining:
                  Remaining Estimate - 24h
                  24h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified