Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-1345 Cache stability issues in load/failover tests
  3. IGNITE-1090

GridCachePartitionedOptimisticTxNodeRestartTest hangs when node lefts topology

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 2.8
    • cache

    Description

      The test hangs with the following errors:

      [16:26:33] (err) Failed to execute compound future reducer: Compound future listener: GridCompoundIdentityFuture [super=GridCompoundFuture [lsnrCalls=0, finished=false, rdc=Map reducer: {}, init=true, res=java.util.concurrent.atomic.AtomicMarkableReference@3c6fff03, err=null, done=false, cancelled=false, err=null, futs=[true]]]class org.apache.ignite.IgniteCheckedException: Failed to wait for topology version to change: AffinityTopologyVersion [topVer=274, minorTopVer=0]
      	at org.apache.ignite.internal.processors.cache.distributed.GridFutureRemapTimeoutObject.onTimeout(GridFutureRemapTimeoutObject.java:64)
      	at org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:158)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: class org.apache.ignite.internal.cluster.ClusterTopologyCheckedException: Remote node left grid (will retry): 703c58e1-e099-4f22-b5c3-6c3676338007
      	at org.apache.ignite.internal.processors.cache.distributed.near.GridNearGetFuture.onNodeLeft(GridNearGetFuture.java:215)
      	at org.apache.ignite.internal.processors.cache.GridCacheMvccManager$3.onEvent(GridCacheMvccManager.java:186)
      	at org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.notifyListeners(GridEventStorageManager.java:745)
      	at org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.notifyListeners(GridEventStorageManager.java:730)
      	at org.apache.ignite.internal.managers.eventstorage.GridEventStorageManager.record(GridEventStorageManager.java:270)
      	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.recordEvent(GridDiscoveryManager.java:1719)
      	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:1910)
      	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:1758)
      	... 2 more
      

      This issue appeared after IGNITE-882 was fixed. Until it wasn't fixed the tests hung because of this issue:

      Caused by: class org.apache.ignite.IgniteCheckedException: Remote node ID is not as expected [expected=9019c94c-84f5-4065-89b1-f16fdf708009, rcvd=707ed613-1b65-4771-ac98-5faa6857e007]
      		at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeHandshake(TcpCommunicationSpi.java:2203)
      		at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2025)
      

      Attachments

        Issue Links

          Activity

            People

              sboikov Semen Boikov
              dmagda Denis A. Magda
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m