Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
4.0.0
Description
In some cases, we skip checking null. For example, the last statement in the following set of queries fails with NPE.
CREATE TABLE `store_sales` (`ss_item_sk` bigint); CREATE TABLE `household_demographics` (`hd_demo_sk` bigint); CREATE TABLE `item` (`i_item_sk` bigint); ALTER TABLE `store_sales` ADD CONSTRAINT `pk_ss` PRIMARY KEY (`ss_item_sk`) DISABLE NOVALIDATE RELY; ALTER TABLE `item` ADD CONSTRAINT `pk_i` PRIMARY KEY (`i_item_sk`) DISABLE NOVALIDATE RELY; ALTER TABLE `store_sales` ADD CONSTRAINT `ss_i` FOREIGN KEY (`ss_item_sk`) REFERENCES `item`(`i_item_sk`) DISABLE NOVALIDATE RELY; EXPLAIN SELECT i_item_sk FROM store_sales, household_demographics, item WHERE ss_item_sk = i_item_sk
The NPE happens with HiveJoinConstraintsRule in the above case.
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: NullPointerException null at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:376) ~[hive-service-4.0.0.jar:4.0.0] at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:214) ~[hive-service-4.0.0.jar:4.0.0] at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:270) ~[hive-service-4.0.0.jar:4.0.0] at org.apache.hive.service.cli.operation.Operation.run(Operation.java:286) ~[hive-service-4.0.0.jar:4.0.0] at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:557) ~[hive-service-4.0.0.jar:4.0.0] at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:542) ~[hive-service-4.0.0.jar:4.0.0] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_275] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_275] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_275] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_275] at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) ~[hive-service-4.0.0.jar:4.0.0] at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) ~[hive-service-4.0.0.jar:4.0.0] at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) ~[hive-service-4.0.0.jar:4.0.0] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_275] at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_275] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899) ~[hadoop-common-3.3.6.jar:?] at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) ~[hive-service-4.0.0.jar:4.0.0] at com.sun.proxy.$Proxy42.executeStatementAsync(Unknown Source) ~[?:?] at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:316) ~[hive-service-4.0.0.jar:4.0.0] at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:652) ~[hive-service-4.0.0.jar:4.0.0] at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1670) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1650) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) ~[hive-service-4.0.0.jar:4.0.0] at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250) ~[hive-exec-4.0.0.jar:4.0.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_275] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_275] at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_275] Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveJoinConstraintsRule.onMatch(HiveJoinConstraintsRule.java:277) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:333) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:542) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:407) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:271) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:74) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:202) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:189) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.executeProgram(CalcitePlanner.java:2446) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.executeProgram(CalcitePlanner.java:2405) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPreJoinOrderingTransforms(CalcitePlanner.java:1943) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1686) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1567) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.calcite.tools.Frameworks.lambda$withPlanner$0(Frameworks.java:131) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:914) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:180) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:126) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1319) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:570) ~[hive-exec-4.0.0.jar:4.0.0] at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13149) ~[hive-exec-4.0.0.jar:4.0.0]
Attachments
Issue Links
- links to