Description
When running queries, I got exception like this:
FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.SparkWork cannot be cast to org.apache.hadoop.hive.ql.plan.TezWork 14/07/31 15:08:53 ERROR ql.Driver: FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.SparkWork cannot be cast to org.apache.hadoop.hive.ql.plan.TezWork java.lang.ClassCastException: org.apache.hadoop.hive.ql.plan.SparkWork cannot be cast to org.apache.hadoop.hive.ql.plan.TezWork at org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.addStatsTask(GenMapRedUtils.java:1419) at org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.isMergeRequired(GenMapRedUtils.java:1645) at org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.processFileSink(GenSparkUtils.java:313) at org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.generateTaskTree(SparkCompiler.java:180) at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:199) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9514) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:207) at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:207) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:413) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:309) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:984) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1049) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:265) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:217) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:427) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:800) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Apparently, GenMapRedUtils::addStatsTask only assumes either MapredWork or TezWork, and since we are introducing SparkWork, this need to be fixed.
Attachments
Attachments
Issue Links
- is duplicated by
-
HIVE-7706 ClassCastException trying to CTAS table
- Resolved