Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
3.4.0
-
None
Description
Currently, we have `dfs.namenode.redundancy.considerLoad` equals true by default, and
dfs.namenode.redundancy.considerLoad.factor equals 2.0 by default.
Think about below situation. when we are doing stress test, we may deploy hdfs client onto the datanode. So, this hdfs client will prefer to write to its local datanode and increase this machine's load. Suppose we have 3 datanodes, the load of them are as below: 5.0, 0.2, 0.3.
The load equals to 5.0 will be excluded when choose datanodes for a block. But actually, it is not slow node when load equals to 5.0 for a machine with 80 cpu cores.
So, we should better add a new configuration entry : `dfs.namenode.redundancy.considerLoad.minLoad` to indicate the mininum factor we will make considerLoad take effect.
Attachments
Issue Links
- links to