Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-6171

Bulk loading to another keyspace with sstableloader throws EOFException

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Not A Problem
    • None
    • Legacy/Tools
    • Linux Mint x64 (Olive)
      Cassandra 1.2.9
      [cqlsh 3.1.7 | Cassandra 1.2.9-1~precise~ppa1 | CQL spec 3.0.0 | Thrift protocol 19.36.0]

    • Normal

    Description

      I'm trying to to load a snapshot of keyspace test to another keyspace, called test2.
      I've got the {{java.io.EOFException}}s below.

      Steps to reproduce:

      # cqlsh
      cqlsh> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
      cqlsh> CREATE TABLE test.test (key varchar PRIMARY KEY, value varchar);
      cqlsh> INSERT INTO test.test (key, value) VALUES ('ryan', 'ryan');
      
      # nodetool --host localhost snapshot
      Requested creating snapshot for: all keyspaces 
      Snapshot directory: 1381309132532
      # mkdir test2
      # mkdir test2/test
      # cp -R /var/lib/cassandra/data/test/test/snapshots/1381309132532/* test2/test/
      # cqlsh
      cqlsh> CREATE KEYSPACE test2 WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
      cqlsh> CREATE TABLE test2.test (key varchar PRIMARY KEY, value varchar);
      cqlsh> 
      
      # sstableloader --verbose --debug --nodes localhost test2/test/
      Streaming revelant part of test2/test/test-test-ic-1-Data.db  to [/127.0.0.1]
      
      progress: [/127.0.0.1 0/1 (0)] [total: 0 - 0MB/s (avg: 0MB/s)]Streaming session to /127.0.0.1 failed
      ERROR 11:02:43,612 Error in ThreadPoolExecutor
      java.lang.RuntimeException: java.io.EOFException
      	at com.google.common.base.Throwables.propagate(Throwables.java:160)
      	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:724)
      Caused by: java.io.EOFException
      	at java.io.DataInputStream.readInt(DataInputStream.java:392)
      	at org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:193)
      	at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:101)
      	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
      	... 3 more
      Exception in thread "Streaming to /127.0.0.1:1" java.lang.RuntimeException: java.io.EOFException
      	at com.google.common.base.Throwables.propagate(Throwables.java:160)
      	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:724)
      Caused by: java.io.EOFException
      	at java.io.DataInputStream.readInt(DataInputStream.java:392)
      	at org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:193)
      	at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:101)
      	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
      	... 3 more
      progress: [/127.0.0.1 1/1 (100)] [total: 100 - 0MB/s (avg: 0MB/s)]Streaming to the following hosts failed:
      [/127.0.0.1]
      # 
      

      The system.log file contains the following:

      ERROR [Thread-4] 2013-10-09 11:02:43,608 CassandraDaemon.java (line 192) Exception in thread Thread[Thread-4,5,main]
      java.lang.AssertionError
      	at org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:168)
      	at org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:138)
      	at org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:238)
      	at org.apache.cassandra.net.IncomingTcpConnection.handleStream(IncomingTcpConnection.java:178)
      	at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:78)
      

      A workaround is renaming the snapshot files too (not just the directory):

      # rename --verbose 's/test-test/test2-test/' test2/test/*
      test2/test/test-test-ic-1-CompressionInfo.db renamed as test2/test/test2-test-ic-1-CompressionInfo.db
      test2/test/test-test-ic-1-Data.db renamed as test2/test/test2-test-ic-1-Data.db
      test2/test/test-test-ic-1-Filter.db renamed as test2/test/test2-test-ic-1-Filter.db
      test2/test/test-test-ic-1-Index.db renamed as test2/test/test2-test-ic-1-Index.db
      test2/test/test-test-ic-1-Statistics.db renamed as test2/test/test2-test-ic-1-Statistics.db
      test2/test/test-test-ic-1-Summary.db renamed as test2/test/test2-test-ic-1-Summary.db
      test2/test/test-test-ic-1-TOC.txt renamed as test2/test/test2-test-ic-1-TOC.txt
      # sstableloader --verbose --debug --nodes localhost test2/test/
      Streaming revelant part of test2/test/test2-test-ic-1-Data.db  to [/127.0.0.1]
      
      progress: [/127.0.0.1 0/1 (0)] [total: 0 - 0MB/s (avg: 0MB/s)]Streaming session to /127.0.0.1 completed (waiting on 0 outstanding sessions)
      progress: [/127.0.0.1 1/1 (100)] [total: 100 - 0MB/s (avg: 0MB/s)]
      # 
      

      sstableloader --help mentions this:

      sstableloader  --help
      usage: sstableloader [options] <dir_path>
      --
      Bulk load the sstables found in the directory <dir_path> to the configured
      cluster.The parent directory of <dir_path> is used as the keyspace name.
      So for instance, to load an sstable named Standard1-g-1-Data.db into
      keyspace Keyspace1, you will need to have the files Standard1-g-1-Data.db
      and Standard1-g-1-Index.db in a directory Keyspace1/Standard1/ in the
      directory and call: sstableloader Keyspace1/Standard1
      

      However, renaming the snapshot files to the suggested format (if I'm right here) still not help:

      # rename --verbose 's/test2-test-/test-/' test2/test/*
      test2/test/test2-test-ic-1-CompressionInfo.db renamed as test2/test/test-ic-1-CompressionInfo.db
      test2/test/test2-test-ic-1-Data.db renamed as test2/test/test-ic-1-Data.db
      test2/test/test2-test-ic-1-Filter.db renamed as test2/test/test-ic-1-Filter.db
      test2/test/test2-test-ic-1-Index.db renamed as test2/test/test-ic-1-Index.db
      test2/test/test2-test-ic-1-Statistics.db renamed as test2/test/test-ic-1-Statistics.db
      test2/test/test2-test-ic-1-Summary.db renamed as test2/test/test-ic-1-Summary.db
      test2/test/test2-test-ic-1-TOC.txt renamed as test2/test/test-ic-1-TOC.txt
      
      # sstableloader --verbose --debug --nodes localhost test2/test/
      Skipping file test-ic-1-Data.db: column family test2.ic doesn't exist
      No sstables to stream
      
      progress: [total: 100 - 0MB/s (avg: 0MB/s)][11:42:53] (0)
      

      An upgraded sstableloader --help message and a helpful error message (instead of the EOFException) would be much better here.

      Attachments

        Activity

          People

            Unassigned Unassigned
            palacsint palacsint
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: