Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-108 Redesign of Ignite Streaming
  3. IGNITE-428

Implement IgniteKafkaStreamer to stream data from Apache Kafka

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • sprint-5
    • sprint-7
    • streaming
    • None

    Description

      We have IgniteDataStreamer which is used to load data into Ignite under high load. It was previously named IgniteDataLoader, see ticket IGNITE-394.

      See Apache Kafka for more information.

      We should create IgniteKafkaStreamer which will consume messages from Kafka and stream them into Ignite caches.

      More details to follow, but to the least we should be able to:

      • Convert Kafka messages to Ignite data using an optional pluggable converter.
      • Specify the cache name for the Ignite cache to load data into.
      • Specify other flags available on IgniteDataStreamer class.

      Attachments

        1. ignite-sprint-5_ignite-428.patch
          33 kB
          Vishal Garg
        2. ignite-sprint-5_ignite-428.patch
          32 kB
          Vishal Garg
        3. ignite-sprint-5_ignite-428.patch
          33 kB
          Vishal Garg

        Activity

          People

            yzhdanov Yakov Zhdanov
            dsetrakyan Dmitriy Setrakyan
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: