Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.8.0
-
None
-
Reviewed
Description
Hit the following NPE when reviewing fix for HDFS-9387 with manual tests as NNThroughputBenchmark currently does not have JUnit tests.
HW11217:centos6.4 xyao$ hadoop org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark -op replication -namenode hdfs://HW11217.local:9000
15/11/12 14:52:03 INFO namenode.NNThroughputBenchmark: Starting benchmark: replication
15/11/12 14:52:03 ERROR namenode.NNThroughputBenchmark: java.lang.NullPointerException
at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark$ReplicationStats.generateInputs(NNThroughputBenchmark.java:1312)
at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark$OperationStatsBase.benchmark(NNThroughputBenchmark.java:280)
at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark.run(NNThroughputBenchmark.java:1509)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark.main(NNThroughputBenchmark.java:1534)
...
However, the root cause is different from HDFS-9387.
From ReplicationStats#generateInputs, nameNode is uninitialized before use, which causes the NPE.
final FSNamesystem namesystem = nameNode.getNamesystem();
From NNThroughputBenchmark#run, nameNode is only initialized when -namenode option is not specified. The fix is to initialize it properly in the else block when -namenode option is specified OR we should block this if it is not supported.
if (namenodeUri == null) { nameNode = NameNode.createNameNode(argv, config); NamenodeProtocols nnProtos = nameNode.getRpcServer(); nameNodeProto = nnProtos; clientProto = nnProtos; dataNodeProto = nnProtos; refreshUserMappingsProto = nnProtos; bpid = nameNode.getNamesystem().getBlockPoolId(); } else { FileSystem.setDefaultUri(getConf(), namenodeUri); DistributedFileSystem dfs = (DistributedFileSystem) FileSystem.get(getConf()); final URI nnUri = new URI(namenodeUri); nameNodeProto = DFSTestUtil.getNamenodeProtocolProxy(config, nnUri, UserGroupInformation.getCurrentUser());