Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-4118

flink write hudi sync hive .If the Hive table already exists, write to create automatically will throw an exception

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.10.1
    • 0.11.1
    • None

    Description

      flink write hudi sync hive 

      If the Hive table already exists, write to create automatically will throw an exception :

      but The program runs normally

      2022-04-21 14:03:12,628 INFO  org.apache.hudi.org.apache.hadoop.hive.metastore.HiveMetaStoreClient [] - Closed a connection to metastore, current connections: 0
      2022-04-21 14:03:12,628 ERROR org.apache.hudi.sink.StreamWriteOperatorCoordinator          [] - Executor executes action [sync hive metadata for instant 20220421140236703] error
      java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.thrift.Type.getType(Lorg/apache/hudi/org/apache/hive/service/rpc/thrift/TTypeId;)Lorg/apache/hadoop/hive/serde2/thrift/Type;
      	at org.apache.hudi.org.apache.hive.service.cli.TypeDescriptor.(TypeDescriptor.java:47) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.org.apache.hive.service.cli.ColumnDescriptor.(ColumnDescriptor.java:46) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.org.apache.hive.service.cli.TableSchema.(TableSchema.java:46) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.org.apache.hive.jdbc.HiveQueryResultSet.retrieveSchema(HiveQueryResultSet.java:264) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.org.apache.hive.jdbc.HiveQueryResultSet.(HiveQueryResultSet.java:198) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.org.apache.hive.jdbc.HiveQueryResultSet$Builder.build(HiveQueryResultSet.java:179) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.org.apache.hive.jdbc.HiveDatabaseMetaData.getColumns(HiveDatabaseMetaData.java:278) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.hive.ddl.JDBCExecutor.getTableSchema(JDBCExecutor.java:129) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.hive.HoodieHiveClient.getTableSchema(HoodieHiveClient.java:225) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.hive.HiveSyncTool.syncSchema(HiveSyncTool.java:248) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:184) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:129) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:115) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.sink.StreamWriteOperatorCoordinator.syncHive(StreamWriteOperatorCoordinator.java:302) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at org.apache.hudi.sink.utils.NonThrownExecutor.lambda$execute$0(NonThrownExecutor.java:93) ~[hudi-flink-bundle_2.11-0.10.1.jar:0.10.1]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_202]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_202]
      	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_202]
      2022-04-21 14:03:12,629 INFO  org.apache.hudi.client.AbstractHoodieClient                  [] - Stopping Timeline service !!
      2022-04-21 14:03:12,629 INFO  org.apache.hudi.client.embedded.EmbeddedTimelineService      [] - Closing Timeline server
      2022-04-21 14:03:12,630 INFO  org.apache.hudi.timeline.service.TimelineService             [] - Closing Timeline Service 

       

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              gaogao110 gaogao
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: