Details
-
Bug
-
Status: Closed
-
Blocker
-
Resolution: Fixed
-
0.18.0
-
None
-
None
-
Reviewed
Description
Currently during name-node startup under-replicated blocks are not added to the neededReplications queue until the name-node leaves safe mode. This is an optimization since otherwise all blocks will first go into the under-replicated queue and then most of them will be removed from it.
When the name-node leaves safe-mode automatically it checks all blocks to have a correct number of replicas (processMisReplicatedBlocks()).
When the name-node leaves safe-mode manually it does not perform the checkup.
In the latter case all under-replicated blocks remain not replicated forever because there is no alternative mechanism to trigger replications.
The proposal is to call processMisReplicatedBlocks() any time the name-node leaves safe mode - automatically or manually.
In addition to solving that problem this could be an alternative mechanism for refreshing neededReplications and excessReplicateMap sets.
Attachments
Attachments
Issue Links
- is related to
-
HDFS-406 Automate test for Hadoop-4597
- Resolved
- relates to
-
HADOOP-4610 Always calculate mis-replicated blocks when safe-mode is turned off.
- Closed