Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-25566

RoundRobinTableInputFormat

    XMLWordPrintableJSON

Details

    • Reviewed
    • Hide
      Adds RoundRobinTableInputFormat, a subclass of TableInputFormat, that takes the TIF#getSplits list and resorts it so as to spread the InputFormats as broadly about the cluster as possible. RRTIF works to frustrate bunching of InputSplits on RegionServers to avoid the scenario where a few RegionServers are working hard fielding many InputSplits while others idle hosting a few or none.
      Show
      Adds RoundRobinTableInputFormat, a subclass of TableInputFormat, that takes the TIF#getSplits list and resorts it so as to spread the InputFormats as broadly about the cluster as possible. RRTIF works to frustrate bunching of InputSplits on RegionServers to avoid the scenario where a few RegionServers are working hard fielding many InputSplits while others idle hosting a few or none.

    Description

      huaxiang figured we needed a round robin on the result returned out TableInputFormat to better distribute task on a production cluster at our place-of-employ.

      This seems to be an old problem. See this helpful, old Flipkart blog: https://tech.flipkart.com/is-data-locality-always-out-of-the-box-in-hadoop-not-really-2ae9c95163cb

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-25662 Fix spotbugs warning in RoundRobinTableInputFormat

          • Major - Major loss of function.
          • Resolved
          relates to

          Task - A task that needs to be done. HBASE-27343 RoundRobinTableInputFormat main should extend `AbstractHBaseTool` for proper configuration parsing

          • Major - Major loss of function.
          • Open
          links to

          Web Link GitHub Pull Request #2947

          Activity

            People

              discoversudhir Sudhir Reddy
              stack Michael Stack
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: