Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-15680

testChangePeersChaosWithSnapshot is flaky

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      testChangePeersChaosWithSnapshot is started to fall with a timeout, in logs we can see:

      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 2021-10-04 19:51:41:503 +0300 [ERROR][%172.17.0.7:5010%JRaft-Common-Executor-0][LocalSnapshotCopier] Fail to start copy job
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] java.nio.channels.ClosedByInterruptException
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at java.base/java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:199)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at java.base/sun.nio.ch.FileChannelImpl.endBlocking(FileChannelImpl.java:162)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at java.base/sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:465)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at org.apache.ignite.raft.jraft.util.Utils.fsync(Utils.java:408)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at org.apache.ignite.raft.jraft.storage.io.MessageFile.save(MessageFile.java:94)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at org.apache.ignite.raft.jraft.storage.snapshot.local.LocalSnapshotMetaTable.saveToFile(LocalSnapshotMetaTable.java:133)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at org.apache.ignite.raft.jraft.storage.snapshot.local.LocalSnapshotWriter.sync(LocalSnapshotWriter.java:103)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at org.apache.ignite.raft.jraft.storage.snapshot.local.LocalSnapshotCopier.copyFile(LocalSnapshotCopier.java:197)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at org.apache.ignite.raft.jraft.storage.snapshot.local.LocalSnapshotCopier.internalCopy(LocalSnapshotCopier.java:123)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at org.apache.ignite.raft.jraft.storage.snapshot.local.LocalSnapshotCopier.startCopy(LocalSnapshotCopier.java:100)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      [19:51:41]W:		 [org.apache.ignite:ignite-raft] 	at java.base/java.lang.Thread.run(Thread.java:834)
      
      

       Seems, that jraft snapshotting mechanism couldn't start to copy for some reason. After that 

       

      There is a chance that the problem has appeared after this temporary solution https://issues.apache.org/jira/browse/IGNITE-15663 

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              maliev Mirza Aliev
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: