Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-3035

Cross platform job submission no supported

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      When the Tez client is located on a windows machine and the Hadoop cluster is located on Linux machines, Job submission fails for Tez (same for MapReduce) with such a exception:

      java.lang.RuntimeException: Job job_1448280010118_1679 failed! Failure info: Application application_1448280010118_1679 failed 2 times due to AM Container for appattempt_1448280010118_1679_000002 exited with  exitCode: 1
      For more detailed output, check application tracking page:http://hmaster.datameer.lan:8088/proxy/application_1448280010118_1679/Then, click on links to logs of each attempt.
      Diagnostics: Exception from container-launch.
      Container id: container_e13_1448280010118_1679_02_000001
      Exit code: 1
      Exception message: /bin/bash: line 0: fg: no job control
      
      Stack trace: ExitCodeException exitCode=1: /bin/bash: line 0: fg: no job control
      	at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
      	at org.apache.hadoop.util.Shell.run(Shell.java:455)
      	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
      	at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      

      For MapReduce i was able to fix the problem by setting the property mapreduce.app-submission.cross-platform=true (see https://issues.apache.org/jira/browse/MAPREDUCE-4052).

      Attachments

        Activity

          People

            Unassigned Unassigned
            oae Johannes Zillmann
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: