Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-182

New Decorator to throttle data and shell command to configure it

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.9.2
    • Sinks+Sources
    • None

    Description

      This feature adds a (sink) decorator called "choke decorator" which has the ability to throttle the bytes going across it. Each such decorator is instantiated using an id, for example to put a choke decorator for a (logical) node1 between source and sink we'll use the following syntax:

      node1 : source |

      {choke(cID)=>sink}

      Where cID is a "choke ID" on the physical node containing to node1.

      The master can put a throttle limit on cID using a new command called "settlimit", the syntax is :

      settlimit physicalNodeName cID max_limit

      This command puts a limit on the cId, which means that all the choke decorators on the physical-node with choke id equal to cID will have a collective throttling limit equal to max_limit.

      One also has ability to put a total throttle limit on the physical-node using the following command :
      settlimit physicalNodeName max_limit.

      The settlimit command is implemented using a new RPC between the Master and the Nodes.

      There is a whole policy stuff to follow (in a future Jira); how to divide the physical node limit between different logical-nodes.

      Attachments

        Issue Links

          Activity

            People

              flume_vibhor Disabled imported user
              flume_vibhor Disabled imported user
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: