Details
Description
Cache interceptor cause problem of missing binary scheme if sql insert contains only entry key fields and no one value field pass in query.
There is exception happened if inside interceptor we try to make BinaryObjectBuilder from entry value BinaryObject (#toBuilder()) and then request builder for any field.
Interceptor example:
new CacheInterceptorAdapter<BinaryObject, BinaryObject>() { @Nullable @Override public BinaryObject onBeforePut(Cache.Entry<BinaryObject, BinaryObject> entry, BinaryObject newVal) { BinaryObjectBuilder newValBuilder = newVal.toBuilder(); String bValue = newValBuilder.getField("B"); // Cannot find schema for object with compact footer // ... } }
It's a pretty serious error which currupt and shutdown grid.
Code example:
LinkedHashMap<String, String> fields = new LinkedHashMap<>(); fields.put("A", "java.lang.String"); fields.put("B", "java.lang.String"); fields.put("C", "java.lang.String"); Set<String> keyFields = new LinkedHashSet<>(); keyFields.add("A"); CacheInterceptorAdapter cacheInterceptorAdapter = new CacheInterceptorAdapter<BinaryObject, BinaryObject>() { @Nullable @Override public BinaryObject onBeforePut(Cache.Entry<BinaryObject, BinaryObject> entry, BinaryObject newVal) { BinaryObjectBuilder newValBuilder = newVal.toBuilder(); String bValue = newValBuilder.getField("B"); // Cannot find schema for object with compact footer if (bValue == null || bValue.isEmpty()) { newValBuilder.setField("B", "Some value"); } return newValBuilder.build(); } }; CacheConfiguration cacheConfiguration = new CacheConfiguration<>() .setName("TEST_CACHE") .setKeyConfiguration(new CacheKeyConfiguration() .setTypeName("TEST_CACHE_KEY") .setAffinityKeyFieldName("InternalId")) .setQueryEntities(Collections.singleton(new QueryEntity() .setTableName("TEST_CACHE") .setKeyType("TEST_CACHE_KEY") .setValueType("TEST_CACHE_VALUE") .setFields(fields) .setKeyFields(keyFields))) .setInterceptor(cacheInterceptorAdapter); IgniteConfiguration igniteConfiguration = new IgniteConfiguration() .setCacheConfiguration(cacheConfiguration); try (Ignite ignite = Ignition.start(igniteConfiguration)) { IgniteCache testCache = ignite.getOrCreateCache("TEST_CACHE"); // putSql testCache.query(new SqlFieldsQuery("INSERT INTO TEST_CACHE (A) VALUES ('1234')")); }
Exception:
[2022-01-18 13:12:32,727][ERROR][main][CacheObjectBinaryProcessorImpl] Timed out while waiting for schema update [typeId=1147851335, schemaId=0] [2022-01-18 13:12:32,730][ERROR][main][root] Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is corrupted [groupId=-838655627, pageIds=[844420635166307], msg=Runtime failure on search row: SearchRow [key=TEST_CACHE_KEY [idHash=100929741, hash=-639470899, A=123], hash=-639470899, cacheId=0]]]] class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is corrupted [groupId=-838655627, pageIds=[844420635166307], msg=Runtime failure on search row: SearchRow [key=TEST_CACHE_KEY [idHash=100929741, hash=-639470899, A=123], hash=-639470899, cacheId=0]] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:6237) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1988) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1742) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1725) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:449) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2331) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2553) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2016) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1833) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1692) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:300) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:481) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:441) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:249) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1147) at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:615) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2571) at org.apache.ignite.internal.processors.cache.GridCacheAdapter.putIfAbsent(GridCacheAdapter.java:3018) at org.apache.ignite.internal.processors.query.h2.dml.DmlUtils.dmlDoInsert(DmlUtils.java:209) at org.apache.ignite.internal.processors.query.h2.dml.DmlUtils.processSelectResult(DmlUtils.java:175) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeUpdateNonTransactional(IgniteH2Indexing.java:2903) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeUpdate(IgniteH2Indexing.java:2748) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeUpdateDistributed(IgniteH2Indexing.java:2674) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeDml(IgniteH2Indexing.java:1260) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1182) at org.apache.ignite.internal.processors.query.GridQueryProcessor$2.applyx(GridQueryProcessor.java:2883) at org.apache.ignite.internal.processors.query.GridQueryProcessor$2.applyx(GridQueryProcessor.java:2879) at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36) at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:3478) at org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$3(GridQueryProcessor.java:2899) at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2934) at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2873) at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2800) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:839) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:787) at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:430) at org.apache.ignite.examples.BBExample.main(BBExample.java:61) Caused by: class org.apache.ignite.binary.BinaryObjectException: Cannot find schema for object with compact footer [typeName=TEST_CACHE_VALUE, typeId=1147851335, missingSchemaId=0, existingSchemaIds=[-2128831035]] at org.apache.ignite.internal.binary.BinaryReaderExImpl.getOrCreateSchema(BinaryReaderExImpl.java:2044) at org.apache.ignite.internal.binary.builder.BinaryBuilderReader.schema(BinaryBuilderReader.java:117) at org.apache.ignite.internal.binary.builder.BinaryObjectBuilderImpl.ensureReadCacheInit(BinaryObjectBuilderImpl.java:497) at org.apache.ignite.internal.binary.builder.BinaryObjectBuilderImpl.getField(BinaryObjectBuilderImpl.java:553) at org.apache.ignite.examples.BBExample$1.onBeforePut(BBExample.java:39) at org.apache.ignite.examples.BBExample$1.onBeforePut(BBExample.java:35) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.update(GridCacheMapEntry.java:6441) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:6243) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:5927) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:4093) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$5200(BPlusTree.java:3987) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:2065) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1955) ... 35 more [2022-01-18 13:12:32,731][ERROR][main][FailureProcessor] A critical problem with in-memory data structures was detected. [2022-01-18 13:12:32,731][ERROR][main][FailureProcessor] No deadlocked threads detected. [2022-01-18 13:12:32,746][ERROR][main][FailureProcessor] Thread dump at 2022/01/18 13:12:32 MSK
Root cause:
The empty binary object is created invalid:
- with invalid schema: zero value insead of `BinaryUtils#schemaInitialId`
- with invalid offset: zero value instead of lenght of the header. It's a cause of invalid modification the empty object by builder.
Attachments
Issue Links
- links to
Activity
ASF GitHub Bot
logged work - 12/May/22 12:06
-
- Time Spent:
- 10m
-
tledkov-gridgain opened a new pull request, #10021:
URL: https://github.com/apache/ignite/pull/10021
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
### The Contribution Checklist
- [ ] There is a single JIRA ticket related to the pull request.
- [ ] The web-link to the pull request is attached to the JIRA ticket.
- [ ] The JIRA ticket has the _Patch Available_ state.
- [ ] The pull request body describes changes that have been made.
The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
- [ ] The pull request title is treated as the final commit message.
The following pattern must be used: `IGNITE-XXXX Change summary` where `XXXX` - number of JIRA issue.
- [ ] A reviewer has been mentioned through the JIRA comments
(see [the Maintainers list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers))
- [ ] The pull request has been checked by the Teamcity Bot and
the `green visa` attached to the JIRA ticket (see [TC.Bot: Check PR](https://mtcga.gridgain.com/prs.html))
### Notes
- [How to Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
- [Coding abbreviation rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
- [Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
- [Apache Ignite Teamcity Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com _#ignite_ channel.
ASF GitHub Bot
logged work - 16/May/22 14:32
-
- Time Spent:
- 10m
-
korlov42 commented on code in PR #10021:
URL: https://github.com/apache/ignite/pull/10021#discussion_r873801370
##########
modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryWriterExImpl.java:
##########
@@ -325,8 +325,8 @@ else if (offsetByteCnt == BinaryUtils.OFFSET_2)
flags |= BinaryUtils.FLAG_HAS_RAW;
}
else {
- finalSchemaId = 0;
Review Comment:
Objects which serialised by using raw writer only are affected by the very same issue. Looks like we don't need the `finalSchemaId` field at all
ASF GitHub Bot
logged work - 17/May/22 15:48
-
- Time Spent:
- 10m
-
korlov42 commented on code in PR #10021:
URL: https://github.com/apache/ignite/pull/10021#discussion_r874989684
##########
modules/core/src/test/java/org/apache/ignite/internal/binary/BinaryMarshallerSelfTest.java:
##########
@@ -6118,4 +6208,22 @@ public TwoCollections() {
v = new Value(127);
}
}
+
+ /** */
+ public static class EmptyBinarylizable implements Binarylizable {
Review Comment:
why didn't you use `ObjectRaw` class instead?
ASF GitHub Bot
logged work - 18/May/22 07:54
-
- Time Spent:
- 10m
-
tledkov-gridgain merged PR #10021:
URL: https://github.com/apache/ignite/pull/10021