Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
Description
The ParseException contains high level information about problems encountered during parsing but currently the stacktrace is pretty shallow.
select * from author where true > fname in ('Alex','Rob')
Currently the query above will fail with the above stacktrace:
org.apache.hadoop.hive.ql.parse.ParseException: line 4:41 cannot recognize input near 'in' '(' ''Alex'' in expression specification
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:125)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:97)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:89)
at org.apache.hadoop.hive.ql.Compiler.parse(Compiler.java:172)
at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:105)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
The end-user gets a hint about what the error might be but the developer has no way to tell how far we went into parsing the given statement and which grammar rule failed to pass.
In some cases we are catching the RecognitionException which shows exactly at which point the parser failed to advance so we could pass it as a cause inside the ParseException. With this change the stacktrace would be similar to the one below:
org.apache.hadoop.hive.ql.parse.ParseException: line 4:41 cannot recognize input near 'in' '(' ''Alex'' in expression specification
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:125)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:97)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:89)
at org.apache.hadoop.hive.ql.Compiler.parse(Compiler.java:172)
at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:105)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
Caused by: NoViableAltException(173@[()* loopback of 739:5: (equal= precedenceEqualOperator p= precedenceSimilarExpression -> ^( $equal $p) |dist= precedenceDistinctOperator p= precedenceSimilarExpression -> ^( KW_NOT["not"] ^( EQUAL_NS["<=>"] $p) ) )*])
at org.antlr.runtime.DFA.noViableAlt(DFA.java:158)
at org.antlr.runtime.DFA.predict(DFA.java:116)
at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceEqualExpression(HiveParser_IdentifiersParser.java:11812)
at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceUnarySuffixExpression(HiveParser_IdentifiersParser.java:12370)
at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceNotExpression(HiveParser_IdentifiersParser.java:12560)
at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceAndExpression(HiveParser_IdentifiersParser.java:12669)
at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceOrExpression(HiveParser_IdentifiersParser.java:12810)
at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.expression(HiveParser_IdentifiersParser.java:8566)
at org.apache.hadoop.hive.ql.parse.HiveParser.expression(HiveParser.java:45010)
at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.searchCondition(HiveParser_FromClauseParser.java:6622)
at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.whereClause(HiveParser_FromClauseParser.java:6540)
at org.apache.hadoop.hive.ql.parse.HiveParser.whereClause(HiveParser.java:45058)
at org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:38576)
at org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:38849)
at org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:38442)
at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:37704)
at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:37592)
at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2763)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1640)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:123)
... 59 more
End-users still get the same high-level error message and developers can see exactly at which point the parser crashed as well as which rule might be worth investigating.
Attachments
Issue Links
- links to