Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-4273

Clear off staging files when TezYarnClient is unable to submit applications

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.10.1, 0.9.3
    • None
    • None

    Description

      Currently it leaves behind few resources like "tez-conf.pb" etc when exception is encountered during app submission. This causes issues in cluster, when apps continue to submit queries continuously.

      drwx------   - hive supergroup          0 2021-01-28 01:58 /tmp/hive/hive/_tez_session_dir/806cd302-abd0-4694-be4a-b2b2473e75a8/.tez/application_1611791897439_0042
      -rw-r--r--   3 hive supergroup     135519 2021-01-28 01:58 /tmp/hive/hive/_tez_session_dir/806cd302-abd0-4694-be4a-b2b2473e75a8/.tez/application_1611791897439_0042/tez-conf.pb
      -rw-r--r--   3 hive supergroup       1056 2021-01-28 01:58 /tmp/hive/hive/_tez_session_dir/806cd302-abd0-4694-be4a-b2b2473e75a8/.tez/application_1611791897439_0042/tez.session.local-resources.pb	
      
      
      
      Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1611791897439_0042 to YARN : org.apache.hadoop.security.AccessControlException: Queue root.default already has X applications from user hive cannot accept submission of application: application_1611791897439_0042
      	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:322) ~[hadoop-yarn-client-3.x:?]
      	at org.apache.tez.client.TezYarnClient.submitApplication(TezYarnClient.java:77) ~[tez-api-0.9.x.jar:0.9.x]
      	at org.apache.tez.client.TezClient.start(TezClient.java:405) ~[tez-api-0.9.x.jar:0.9.x]
      	at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:535) ~[hive-exec-3.x]

      Attachments

        1. TEZ-4273.1.patch
          2 kB
          Rajesh Balamohan

        Activity

          People

            rajesh.balamohan Rajesh Balamohan
            rajesh.balamohan Rajesh Balamohan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: