Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-11346

Hardcoded absolute path for bash and mv command

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.3.4
    • None
    • nodemanager
    • None

    Description

      Not all operating systems have bash and mv in /bin folder. Seems that in HADOOP-13457 and HADOOP-13797 it is already aware but currently the issue still exists in hadoop-yarn.

      For example, in NixOS, bash and mv is in /nix/store/..., linked to /run/current-system/sw/bin. In this case, when running:

      ./bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.4.jar grep input output 'dfs[a-z.]+'

      It will fail when running default_container_executor.sh with the following message:

      Exception message: .../nm-local-dir/usercache/weko/appcache/application_1665637385308_0007/container_1665637385308_0007_02_000001/ .sh: line 2: /bin/bash: No such file or directory
      .../nm-local-dir/usercache/weko/appcache/application_1665637385308_0007/container_1665637385308_0007_02_000001/default_container_executor.sh: line 5: /bin/mv: No such file or directory

      because the script cannot find /bin/bash and /bin/mv.

      For your information, "/usr/bin/env bash", which is more generic, can be used to replace "/bin/bash". It also works for "/bin/mv".

      Attachments

        1. hadoop-conf.zip
          41 kB
          wy
        2. image-2022-10-13-14-21-52-653.png
          31 kB
          wy

        Activity

          People

            Unassigned Unassigned
            wekoms wy
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: