Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-22578

hive2 queries fails after adding any service to the cluster

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 2.6.0
    • 2.6.1
    • None
    • None

    Description

      After adding beacon component, hive2 queries fail with below exception

       select t_hour,count(t_hour) from time_dim group by t_hour;
      INFO  : Compiling command(queryId=hive_20171117151724_678c0fad-bb02-4a07-9b5b-2f8c6ad34013): select t_hour,count(t_hour) from time_dim group by t_hour
      INFO  : We are setting the hadoop caller context from HIVE_SSN_ID:3d33e44f-5f12-40d1-a88c-8cb8e7841885 to hive_20171117151724_678c0fad-bb02-4a07-9b5b-2f8c6ad34013
      INFO  : Semantic Analysis Completed
      INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:t_hour, type:int, comment:null), FieldSchema(name:_c1, type:bigint, comment:null)], properties:null)
      INFO  : Completed compiling command(queryId=hive_20171117151724_678c0fad-bb02-4a07-9b5b-2f8c6ad34013); Time taken: 0.131 seconds
      INFO  : We are resetting the hadoop caller context to HIVE_SSN_ID:3d33e44f-5f12-40d1-a88c-8cb8e7841885
      INFO  : Setting caller context to query id hive_20171117151724_678c0fad-bb02-4a07-9b5b-2f8c6ad34013
      INFO  : Executing command(queryId=hive_20171117151724_678c0fad-bb02-4a07-9b5b-2f8c6ad34013): select t_hour,count(t_hour) from time_dim group by t_hour
      INFO  : Query ID = hive_20171117151724_678c0fad-bb02-4a07-9b5b-2f8c6ad34013
      INFO  : Total jobs = 1
      INFO  : Launching Job 1 out of 1
      INFO  : Starting task [Stage-1:MAPRED] in serial mode
      INFO  : Tez session hasn't been created yet. Opening session
      INFO  : Dag name: select t_hour,count(t_hour) from ti...t_hour(Stage-1)
      INFO  : Dag submit failed due to Invalid TaskLaunchCmdOpts defined for Vertex Map 1 : Invalid/conflicting GC options found, cmdOpts="-server -Djava.net.preferIPv4Stack=true -Dhdp.version=2.6.3.0-222 -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/grid/0/dumps/hive -server -Xmx11469m -Djava.net.preferIPv4Stack=true -XX:NewRatio=8 -XX:+UseNUMA -XX:+UseParallelGC -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA " stack trace: [org.apache.tez.dag.api.DAG.createDag(DAG.java:1009), org.apache.tez.client.TezClientUtils.prepareAndCreateDAGPlan(TezClientUtils.java:720), org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:555), org.apache.tez.client.TezClient.submitDAG(TezClient.java:522), org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:548), org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:198), org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199), org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100), org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987), org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667), org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414), org.apache.hadoop.hive.ql.Driver.run(Driver.java:1211), org.apache.hadoop.hive.ql.Driver.run(Driver.java:1204), org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242), org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91), org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:336), java.security.AccessController.doPrivileged(Native Method), javax.security.auth.Subject.doAs(Subject.java:422), org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866), org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:350), java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511), java.util.concurrent.FutureTask.run(FutureTask.java:266), java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511), java.util.concurrent.FutureTask.run(FutureTask.java:266), java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142), java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617), java.lang.Thread.run(Thread.java:745)] retrying...
      ERROR : Failed to execute tez graph.
      org.apache.tez.dag.api.TezUncheckedException: Invalid TaskLaunchCmdOpts defined for Vertex Map 1 : Invalid/conflicting GC options found, cmdOpts="-server -Djava.net.preferIPv4Stack=true -Dhdp.version=2.6.3.0-222 -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/grid/0/dumps/hive -server -Xmx11469m -Djava.net.preferIPv4Stack=true -XX:NewRatio=8 -XX:+UseNUMA -XX:+UseParallelGC -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA "
      	at org.apache.tez.dag.api.DAG.createDag(DAG.java:1009) ~[tez-api-0.8.4.2.6.3.0-222.jar:0.8.4.2.6.3.0-222]
      	at org.apache.tez.client.TezClientUtils.prepareAndCreateDAGPlan(TezClientUtils.java:720) ~[tez-api-0.8.4.2.6.3.0-222.jar:0.8.4.2.6.3.0-222]
      	at org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:555) ~[tez-api-0.8.4.2.6.3.0-222.jar:0.8.4.2.6.3.0-222]
      	at org.apache.tez.client.TezClient.submitDAG(TezClient.java:522) ~[tez-api-0.8.4.2.6.3.0-222.jar:0.8.4.2.6.3.0-222]
      	at org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:563) ~[hive-exec-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:198) [hive-exec-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414) [hive-exec-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1211) [hive-exec-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1204) [hive-exec-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242) [hive-service-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91) [hive-service-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:336) [hive-service-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
      	at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_112]
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) [hadoop-common-2.7.3.2.6.3.0-222.jar:?]
      	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:350) [hive-service-2.1.0.2.6.3.0-222.jar:2.1.0.2.6.3.0-222]
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_112]
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_112]
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_112]
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_112]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112]
      	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
      Caused by: org.apache.tez.dag.api.TezException: Invalid/conflicting GC options found, cmdOpts="-server -Djava.net.preferIPv4Stack=true -Dhdp.version=2.6.3.0-222 -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/grid/0/dumps/hive -server -Xmx11469m -Djava.net.preferIPv4Stack=true -XX:NewRatio=8 -XX:+UseNUMA -XX:+UseParallelGC -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA "
      	at org.apache.tez.common.JavaOptsChecker.checkOpts(JavaOptsChecker.java:81) ~[tez-api-0.8.4.2.6.3.0-222.jar:0.8.4.2.6.3.0-222]
      	at org.apache.tez.client.TezClientUtils.addDefaultsToTaskLaunchCmdOpts(TezClientUtils.java:780) ~[tez-api-0.8.4.2.6.3.0-222.jar:0.8.4.2.6.3.0-222]
      	at org.apache.tez.dag.api.DAG.createDag(DAG.java:1005) ~[tez-api-0.8.4.2.6.3.0-222.jar:0.8.4.2.6.3.0-222]
      	... 26 more
      ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
      INFO  : Resetting the caller context to HIVE_SSN_ID:3d33e44f-5f12-40d1-a88c-8cb8e7841885
      INFO  : Completed executing command(queryId=hive_20171117151724_678c0fad-bb02-4a07-9b5b-2f8c6ad34013); Time taken: 52.891 seconds
      Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask (state=08S01,code=1)
      

      Attachments

        1. AMBARI-22578.patch
          1 kB
          Jaimin Jetly

        Issue Links

          Activity

            People

              jaimin Jaimin Jetly
              santhoshbg Santhosh B Gowda
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: