Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-3109 Overhaul the deployment and testing modules
  3. BIGTOP-3110

[Puppet] Hive metastore can not startup successfully

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.0
    • 1.4.0
    • deployment, hive
    • None

    Description

      Using either vagrant or docker provisioner with hive component configured to deploy, though the deployment seems successful, hive commands can not be executed:

      
      hive> show tables;
      FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
      
      

       

      Looking further into the log /var/log/hive/hive-metastore.log

      
      2018-12-13T07:00:17,388 WARN [main] metastore.MetaStoreDirectSql: Self-test query [select "DB_ID" from "DBS"] failed; direct SQL is disabled
      javax.jdo.JDODataStoreException: Error executing SQL query "select "DB_ID" from "DBS"".
       at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543) ~[datanucleus-api-jdo-4.2.4.jar:?]
       at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391) ~[datanucleus-api-jdo-4.2.4.jar:?]
       at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:216) ~[datanucleus-api-jdo-4.2.4.jar:?]
       at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.runTestQuery(MetaStoreDirectSql.java:224) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java:144) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.ObjectStore.initializeHelper(ObjectStore.java:408) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:340) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:301) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) [hadoop-common-2.8.4.jar:?]
       at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) [hadoop-common-2.8.4.jar:?]
       at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:624) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:590) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:584) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:651) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:427) [hive-exec-2.3.3.jar:2.3.3]
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_191]
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_191]
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_191]
       at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
       at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:79) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:92) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6887) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6882) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:7140) [hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:7067) [hive-exec-2.3.3.jar:2.3.3]
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_191]
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_191]
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_191]
       at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
       at org.apache.hadoop.util.RunJar.run(RunJar.java:234) [hadoop-common-2.8.4.jar:?]
       at org.apache.hadoop.util.RunJar.main(RunJar.java:148) [hadoop-common-2.8.4.jar:?]
      Caused by: java.sql.SQLSyntaxErrorException: Table/View 'DBS' does not exist.
      
      ...
      
      2018-12-13T07:00:17,501 ERROR [main] metastore.HiveMetaStore: Metastore Thrift Server threw an exception...
      org.apache.hadoop.hive.metastore.api.MetaException: Version information not found in metastore.
       at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:83) ~[hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:92) ~[hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6887) ~[hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6882) ~[hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:7140) ~[hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:7067) [hive-exec-2.3.3.jar:2.3.3]
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_191]
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_191]
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_191]
       at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
       at org.apache.hadoop.util.RunJar.run(RunJar.java:234) [hadoop-common-2.8.4.jar:?]
       at org.apache.hadoop.util.RunJar.main(RunJar.java:148) [hadoop-common-2.8.4.jar:?]
      Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Version information not found in metastore.
       at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7564) ~[hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:7542) ~[hive-exec-2.3.3.jar:2.3.3]
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_191]
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_191]
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_191]
       at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
       at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101) ~[hive-exec-2.3.3.jar:2.3.3]
       at com.sun.proxy.$Proxy23.verifySchema(Unknown Source) ~[?:?]
       at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:591) ~[hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:584) ~[hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:651) ~[hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:427) ~[hive-exec-2.3.3.jar:2.3.3]
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_191]
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_191]
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_191]
       at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
       at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148) ~[hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) ~[hive-exec-2.3.3.jar:2.3.3]
       at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:79) ~[hive-exec-2.3.3.jar:2.3.3]
       ... 11 more
      2018-12-13T07:00:17,761 INFO [pool-2-thread-1] metastore.HiveMetaStore: Shutting down hive metastore.
      2018-12-13T07:00:17,762 INFO [pool-2-thread-1] metastore.HiveMetaStore: SHUTDOWN_MSG:
      /************************************************************
      SHUTDOWN_MSG: Shutting down HiveMetaStore at bigtop1.vagrant/10.10.10.11
      ************************************************************/
      
      

      Attachments

        Issue Links

          Activity

            People

              evans_ye Evans Ye
              evans_ye Evans Ye
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: