Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-3112

KafkaConnect can not sync to Hive

    XMLWordPrintableJSON

Details

    Description

      Right now KafkaConnect can not sync to Hive.
      KafkaConnect use org.apache.hudi.DataSourceUtils to build HiveSyncConfig now, but in DataSourceUtils class import some spark dependencies. So that KafkaConnect will fail because of the application of related classes.

      Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.types.DataType
	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 66 more

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HUDI-3199 Hive sync config unification

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #4458

          Activity

            People

              Unassigned Unassigned
              chenxiang cdmikechen
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: