Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
In the method read(byte buf[ ], int off, int len) of DFSInputStream, when read fails, it calls "blockSeekTo" to reselect a datanode. However, the failed datanode does not feed back to blockSeekTo. The datanode selection algorithm works as follows:
- If the machine that the client is running on has a local copy, return the local machine;
- Otherwise, randomly pick up one location.
When the failed data node info does not feed back to target reselection, this leads to two flaws:
1. When a client fails to read from the local copy, for example, because of the checksum error, the local machine will always be chosen in retries.
2. Random selection may still return the same failed node.