Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Based on prototyping carried out within IGNITE-15374 it's requited to rework distributed configuration flow processing:
- Use configuration registry instead of meta storage manager.
- Use configuration manager for internal configuration processing. Blocked by
IGNITE-15047 - Use configuration manager in order to retrieve distributed up-to-date values.
- Use configuration listeners instead of meta storage watches.
- Use configuration manager for internal configuration processing. Blocked by
- Favor one-step table creation (calculate assignments and inner schema together with initial table configuration update).
Please pay attention that during prototyping plenty of bugs were found that should be fixed during refactoring. To name some:
- Columns validation is skipped if column was added or removed during alterTable() See onConfigurationChanged() for more details.
if (!newTbl.columns().namedListKeys().equals(oldTbl.columns().namedListKeys())) return true;
- table identified ignores update counter
UUID tblId = new UUID(ctx.storageRevision(), 0L);
Of course in addition to fixing known issues we will introduce new bugs during refactoring.
Following items were implemented:
1. Unnecessary phases of assignment calculation and schema preparation as a reaction on distributed event were simplified to in-place calculation within table creation:
clusterCfgMgr .configurationRegistry() .getConfiguration(TablesConfiguration.KEY) .tables() .change( change -> change.create( name, (ch) -> { tableInitChange.accept(ch); ((ExtendedTableChange)ch). // Table id specification. changeId(tblId.toString()). // Affinity assignments calculation. changeAssignments( ByteUtils.toBytes( AffinityService.calculateAssignments( baselineMgr.nodes(), ch.partitions(), ch.replicas() ) ) ). // Table schema preparation. changeSchemas( schemasCh -> schemasCh.create( String.valueOf(INITIAL_SCHEMA_VERSION), schemaCh -> schemaCh.changeSchema( ByteUtils.toBytes( SchemaService.prepareSchemaDescriptor( ((ExtendedTableView)ch).schemas().size(), ch ) ) ) ) ); } ) )
Besides obvious that allowed to simplify AffinityManager and SchemaManager that were converted to simple stateless services.
2. Same is about table id: instead of using meta storage revision and update counter (this wasn't implemented though) explicit table id generation was used. This is the main reason for the large number of changed files.
3. Inner local events were substituted by common Futures within create/alter/dropTable.
Few more tickets were created for further improvements:
IGNITE-15414 Schema validation refactoring with configuration validators.
IGNITE-15485 Support table rename operation.
Some tickets for further refactoring are still awaited:
IGNITE-15409 Listener with any placeholders
IGNITE-15412 Configuration registry: add ability to retrieve distributed value That will allow to remove metastorage from table manager completely and get rid of current verison of TableManager#tableNamesConfigured
Attachments
Issue Links
- blocks
-
IGNITE-15096 Schema events processing refactoring.
- Resolved
- is related to
-
IGNITE-15374 Prototype event flow refactoring
- Resolved
- links to