Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-6981

multiple commit logs per region server

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • None
    • None
    • None
    • None

    Description

      For write dominated workloads, the fact that we have only 1 commit log per server artificially limits the aggregate throughput to that of one disk (this is especially true if we do true FS level syncs on each datanode, or even frequent intermittent FS level syncs as data is being written to each block).

      We could consider allowing a configurable number of commit logs per server (perhaps something close to or slightly less than the number of disks per server), and we could shard regions on the server, by maybe just a simple modulo scheme, into those commit logs.

      [A quick way to experiment with the same might be to run multiple region server instances per server; but might be better operationally to package the feature into a single region server.]

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              kannanm Kannan Muthukkaruppan
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: