Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-5637

Hive sync using run_sync_tool fails due to NoClassDefFoundError

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 0.13.0
    • None

    Description

      run_sync_tool.sh \
      >   --jdbc-url jdbc:hive2://hiveserver:10000 \
      >   --user hive \
      >   --pass hive \
      >   --partitioned-by dt \
      >   --base-path /user/hive/warehouse/stock_ticks_cow \
      >   --database default \
      >   --table stock_ticks_cow \
      >   --partition-value-extractor org.apache.hudi.hive.SlashEncodedDayPartitionValueExtractor

       

      throws following exception

      2023-01-28 06:31:18,143 INFO  [main] hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 0
      Exception in thread "main" java.lang.NoClassDefFoundError: com/esotericsoftware/kryo/KryoSerializable
          at java.lang.ClassLoader.defineClass1(Native Method)
          at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
          at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
          at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
          at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
          at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
          at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
          at java.security.AccessController.doPrivileged(Native Method)
          at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
          at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
          at org.apache.hudi.common.table.TableSchemaResolver.hasOperationField(TableSchemaResolver.java:481)
          at org.apache.hudi.util.Lazy.get(Lazy.java:54)
          at org.apache.hudi.common.table.TableSchemaResolver.getTableSchemaFromLatestCommitMetadata(TableSchemaResolver.java:231)
          at org.apache.hudi.common.table.TableSchemaResolver.getTableAvroSchemaInternal(TableSchemaResolver.java:199)
          at org.apache.hudi.common.table.TableSchemaResolver.getTableAvroSchema(TableSchemaResolver.java:139)
          at org.apache.hudi.common.table.TableSchemaResolver.getTableParquetSchema(TableSchemaResolver.java:179)
          at org.apache.hudi.sync.common.HoodieSyncClient.getStorageSchema(HoodieSyncClient.java:109)
          at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:234)
          at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:173)
          at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:161)
          at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:420)
      Caused by: java.lang.ClassNotFoundException: com.esotericsoftware.kryo.KryoSerializable
          at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
          at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
          ... 23 more 

      Works with version 0.12.2.

      Attachments

        Issue Links

          Activity

            People

              codope Sagar Sumit
              codope Sagar Sumit
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: