Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-3962

Cassandra parallel data loader

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • cassandra
    • None

    Description

      There should be an option to warm up Ignite caches backed by Cassandra using loadCache method and distributing download process among all the nodes of Ignite cluster - downloading portion of data from Cassandra in parallel.

      Each Ignite node responsible for a set of Ignite cache partitions. By providing some custom mapping structure to loadCache method it's rather easy to map Ignite partitions to "subpartitions" of Cassandra data slice which we are interested in. Such a way it will be possible to download all the data in parallel by utilizing all Ignite cluster nodes and running multiple download thread on each node.

      Attachments

        Issue Links

          Activity

            People

              irudyak Igor Rudyak
              irudyak Igor Rudyak
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: