Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.3.4
-
None
-
None
-
Used Hadoop 3.3.4 downloaded from official site. Configs are in the attachment.
OS: NixOS-WSL (https://github.com/nix-community/NixOS-WSL)
JDK version: 8
Description
Not all operating systems have bash and mv in /bin folder. Seems that in HADOOP-13457 and HADOOP-13797 it is already aware but currently the issue still exists in hadoop-yarn.
For example, in NixOS, bash and mv is in /nix/store/..., linked to /run/current-system/sw/bin. In this case, when running:
./bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.4.jar grep input output 'dfs[a-z.]+'
It will fail when running default_container_executor.sh with the following message:
Exception message: .../nm-local-dir/usercache/weko/appcache/application_1665637385308_0007/container_1665637385308_0007_02_000001/ .sh: line 2: /bin/bash: No such file or directory
.../nm-local-dir/usercache/weko/appcache/application_1665637385308_0007/container_1665637385308_0007_02_000001/default_container_executor.sh: line 5: /bin/mv: No such file or directory
because the script cannot find /bin/bash and /bin/mv.
For your information, "/usr/bin/env bash", which is more generic, can be used to replace "/bin/bash". It also works for "/bin/mv".