Details
Description
Steps:
Launch wordcount example with pipe
/usr/hdp/current/hadoop-client/bin/hadoop pipes "-Dhadoop.pipes.java.recordreader=true" "-Dhadoop.pipes.java.recordwriter=true" -input pipeInput -output pipeOutput -program bin/wordcount
The application fails with below stacktrace
AM
attempt_1517534613368_0041_r_000000_2 is : 0.0 2018-02-02 02:40:51,071 ERROR [IPC Server handler 16 on 43391] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1517534613368_0041_r_000000_2 - exited : java.io.FileNotFoundException: /grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1517534613368_0041/jobTokenPassword (Permission denied) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(FileOutputStream.java:270) at java.io.FileOutputStream.<init>(FileOutputStream.java:213) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:236) at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:219) at org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:318) at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:307) at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:338) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:401) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:464) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:443) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1169) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1149) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1038) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1026) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:703) at org.apache.hadoop.mapred.pipes.Application.writePasswordToLocalFile(Application.java:173) at org.apache.hadoop.mapred.pipes.Application.<init>(Application.java:109) at org.apache.hadoop.mapred.pipes.PipesReducer.startApplication(PipesReducer.java:87) at org.apache.hadoop.mapred.pipes.PipesReducer.reduce(PipesReducer.java:65) at org.apache.hadoop.mapred.pipes.PipesReducer.reduce(PipesReducer.java:38) at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:445) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:393) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Attachments
Attachments
Issue Links
- is broken by
-
MAPREDUCE-7036 ASF License warning in hadoop-mapreduce-client
- Resolved