Details
Major Description
In case of using Ignite Native Persistence, the attempt to insert a new entry which size is greater than WAL buffer size leads to CorruptedTreeException:
org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is corrupted [groupId=-1407396309, pageIds=[1125904201809926], msg=Runtime failure on search row: SearchRow [key=KeyCacheObjectImpl [part=1, val=1, hasValBytes=true], hash=1, cacheId=0]] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:6487) ~[classes/:?] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:2202) ~[classes/:?] at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1698) ~[classes/:?] at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1681) ~[classes/:?] at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:2762) ~[classes/:?] at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:425) ~[classes/:?] at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:1975) ~[classes/:?] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2554) ~[classes/:?] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2014) ~[classes/:?] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1833) ~[classes/:?] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1706) ~[classes/:?] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:300) ~[classes/:?] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:481) ~[classes/:?] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:441) ~[classes/:?] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:249) ~[classes/:?] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1151) ~[classes/:?] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:619) ~[classes/:?] at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2487) ~[classes/:?] at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2466) ~[classes/:?] at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1332) ~[classes/:?] at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:867) ~[classes/:?] at org.apache.ignite.internal.processors.database.IgniteDbPutGetAbstractTest.lambda$testPutLargeEntry$4bb74d57$1(IgniteDbPutGetAbstractTest.java:331) ~[test-classes/:?] at org.apache.ignite.internal.util.lang.RunnableX.run(RunnableX.java:37) ~[classes/:?] at org.apache.ignite.testframework.GridTestUtils.lambda$assertThrows$0(GridTestUtils.java:467) ~[test-classes/:?] at org.apache.ignite.testframework.GridTestUtils.assertThrows(GridTestUtils.java:505) ~[test-classes/:?] at org.apache.ignite.testframework.GridTestUtils.assertThrows(GridTestUtils.java:485) ~[test-classes/:?] at org.apache.ignite.testframework.GridTestUtils.assertThrows(GridTestUtils.java:466) ~[test-classes/:?] at org.apache.ignite.internal.processors.database.IgniteDbPutGetAbstractTest.testPutLargeEntry(IgniteDbPutGetAbstractTest.java:329) ~[test-classes/:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_341] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_341] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_341] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_341] at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) ~[junit-4.12.jar:4.12] at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) ~[junit-4.12.jar:4.12] at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) ~[junit-4.12.jar:4.12] at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) ~[junit-4.12.jar:4.12] at org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2445) ~[test-classes/:?] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_341] Caused by: java.lang.IllegalArgumentException: Record is too long [capacity=16777216, size=33554524] at org.apache.ignite.internal.processors.cache.persistence.wal.SegmentedRingByteBuffer.offer0(SegmentedRingByteBuffer.java:219) ~[classes/:?] at org.apache.ignite.internal.processors.cache.persistence.wal.SegmentedRingByteBuffer.offer(SegmentedRingByteBuffer.java:195) ~[classes/:?] at org.apache.ignite.internal.processors.cache.persistence.wal.filehandle.FileWriteHandleImpl.addRecord(FileWriteHandleImpl.java:234) ~[classes/:?] at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.log(FileWriteAheadLogManager.java:935) ~[classes/:?] at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.log(FileWriteAheadLogManager.java:888) ~[classes/:?] at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.logUpdate(GridCacheMapEntry.java:4006) ~[classes/:?] at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.update(GridCacheMapEntry.java:6172) ~[classes/:?] at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:5918) ~[classes/:?] at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:5603) ~[classes/:?] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:4307) ~[classes/:?] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$5700(BPlusTree.java:4201) ~[classes/:?] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:2279) ~[classes/:?] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:2169) ~[classes/:?] ... 36 more
The root cause is that the new data record is greater than the WAL buffer size, and the IllegalArgumentException exception is not properly handled.
The issue can be solved in the following way:
- if the primary node cannot write a new WAL record (DataRecord actually), it should throw CacheException to a user. Data must be consistent, and neither the primary node, nor the backup node must apply the new update.
- if the primary node successfully wrote a new record to the WAL, but the backup node is unable to write the corresponding record (due to a different WAL buffer size configuration, for example) than the backup node must trigger a configured failure handler to avoid data inconsistencies.
Attachments
Issue Links
- links to
