Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
This feature adds a (sink) decorator called "choke decorator" which has the ability to throttle the bytes going across it. Each such decorator is instantiated using an id, for example to put a choke decorator for a (logical) node1 between source and sink we'll use the following syntax:
node1 : source |
{choke(cID)=>sink}Where cID is a "choke ID" on the physical node containing to node1.
The master can put a throttle limit on cID using a new command called "settlimit", the syntax is :
settlimit physicalNodeName cID max_limit
This command puts a limit on the cId, which means that all the choke decorators on the physical-node with choke id equal to cID will have a collective throttling limit equal to max_limit.
One also has ability to put a total throttle limit on the physical-node using the following command :
settlimit physicalNodeName max_limit.
The settlimit command is implemented using a new RPC between the Master and the Nodes.
There is a whole policy stuff to follow (in a future Jira); how to divide the physical node limit between different logical-nodes.
Attachments
Issue Links
- is related to
-
FLUME-64 Basic Throttling in a FlumeNode
- Closed