Details
Description
Currently, Ignite choose RocksDB-6.20.3 and let all RocksDB table storages share same WriteBufferManager within a data region. This could limit the memory of all storage instances in a way, however, this is still not enough considerring the case that current WriteBufferManager cannot not limit the memory even memory_usage exceeds buffer_size with multi rocksdb instances.
We can refer to this feature descriptions:
"When WriteBufferManager is shared across DBs and column families
to maintain memory usage under a limit, OOMs have been observed when flush cannot
finish but writes continuously insert to memtables.
In order to avoid OOMs, when memory usage goes beyond buffer_limit_ and DBs tries to write,
this change will stall incoming writers until flush is completed and memory_usage
drops." (see the PR summary)
RocksDB introduce better memory management and public it to java side from RocksDB-6.27.3 (see commit-dc00e4b120) Thus, I suggest to bump the RocksDB version to 6.27.3 and enable this feature by default.
Attachments
Issue Links
- links to