Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-108 Redesign of Ignite Streaming
  3. IGNITE-529

Implement IgniteFlumeStreamer to stream data from Apache Flume

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.8
    • streaming
    • None

    Description

      We have IgniteDataStreamer which is used to load data into Ignite under high load. It was previously named IgniteDataLoader, see ticket IGNITE-394.

      See Apache Flume for more information.

      We should create IgniteFlumeStreamer which will consume messages from Apache Flume and stream them into Ignite caches.

      More details to follow, but to the least we should be able to:

      • Convert Flume data to Ignite data using an optional pluggable converter.
      • Specify the cache name for the Ignite cache to load data into.
      • Specify other flags available on IgniteDataStreamer class.

      Attachments

        1. IgniteSinkSetup.txt
          3 kB
          Roman Shtykh

        Activity

          People

            shroman Roman Shtykh
            dsetrakyan Dmitriy Setrakyan
            Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 40m
                40m