Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
2.6.0
-
None
-
None
-
hadoop 2.6.0 hive 1.1.0 tez0.7 cenos6.4
Description
sWhen I enable hdfs federation.I run a query on hive on tez. Then it occur a exception:
8.784 PM WARN org.apache.hadoop.security.UserGroupInformation No groups available for user tangjijun 3:12:28.784 PM ERROR org.apache.hadoop.hive.ql.exec.Task Failed to execute tez graph. java.lang.ArrayIndexOutOfBoundsException: 0 at org.apache.hadoop.fs.viewfs.ViewFileSystem$InternalDirOfViewFs.getFileStatus(ViewFileSystem.java:771) at org.apache.hadoop.fs.viewfs.ViewFileSystem.getFileStatus(ViewFileSystem.java:359) at org.apache.tez.client.TezClientUtils.checkAncestorPermissionsForAllUsers(TezClientUtils.java:955) at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:184) at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:787) at org.apache.tez.client.TezClient.start(TezClient.java:337) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:191) at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:234) at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:136) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1640) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1399) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1183) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:144) at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:69) at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:196) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
I digging into the issue,I found the code snippet in ViewFileSystem.java as follows:
@Override
public FileStatus getFileStatus(Path f) throws IOException {
checkPathIsSlash(f);
return new FileStatus(0, true, 0, 0, creationTime, creationTime,
PERMISSION_555, ugi.getUserName(), ugi.getGroupNames()[0],
new Path(theInternalDir.fullPath).makeQualified(
myUri, ROOT_PATH));
}
If the node in cluster haven't creat user like tangjijun,ugi.getGroupNames()[0] will throw ArrayIndexOutOfBoundsException.Because no user mean no group.
I create user tangjijun on that node. Then the job was executed normally.
I think this code should check ugi.getGroupNames() is empty.When it is empty ,then print some log. Not to throw exception.
Attachments
Attachments
Issue Links
- duplicates
-
HADOOP-13442 Optimize UGI group lookups
-
- Resolved
-