Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-1524

getDAGStatus seems to fork out the entire JVM on non-secure clusters

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.5.0
    • 0.5.1
    • None
    • None

    Description

      Tracked down a consistent fork() call to

      	at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
      	at org.apache.hadoop.util.Shell.run(Shell.java:418)
      	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
      	at org.apache.hadoop.util.Shell.execCommand(Shell.java:739)
      	at org.apache.hadoop.util.Shell.execCommand(Shell.java:722)
      	at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:83)
      	at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:52)
      	at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:50)
      	at org.apache.hadoop.security.Groups.getGroups(Groups.java:139)
      	at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1409)
      	at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getRPCUserGroups(DAGClientAMProtocolBlockingPBServerImpl.java:75)
      	at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.getDAGStatus(DAGClientAMProtocolBlockingPBServerImpl.java:102)
      	at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7375)
      

      hitesh - would it make sense to cache this at all?

      Attachments

        1. TEZ-1524.1.patch
          8 kB
          Gopal Vijayaraghavan
        2. TEZ-1524.2.patch
          38 kB
          Gopal Vijayaraghavan
        3. TEZ-1524.3.patch
          38 kB
          Gopal Vijayaraghavan

        Activity

          People

            gopalv Gopal Vijayaraghavan
            gopalv Gopal Vijayaraghavan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: