Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Duplicate
-
None
-
None
-
None
-
None
Description
For write dominated workloads, the fact that we have only 1 commit log per server artificially limits the aggregate throughput to that of one disk (this is especially true if we do true FS level syncs on each datanode, or even frequent intermittent FS level syncs as data is being written to each block).
We could consider allowing a configurable number of commit logs per server (perhaps something close to or slightly less than the number of disks per server), and we could shard regions on the server, by maybe just a simple modulo scheme, into those commit logs.
[A quick way to experiment with the same might be to run multiple region server instances per server; but might be better operationally to package the feature into a single region server.]
Attachments
Issue Links
- duplicates
-
HBASE-5699 Run with > 1 WAL in HRegionServer
- Closed