Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-20606

sql cli with hive catalog cannot create function using user classes from jar which specified by -j option

    XMLWordPrintableJSON

Details

    Description

      with flink version 1.12.0(versions before also affected)

      I started a sql cli  with a hive catalog and specified a user jar file with -j option like this:

      bin/sql-client.sh embedded -j /Users/akis/Desktop/flink-func/myfunc.jar
      

      when i tried to create a custom function using class from myfunc.jar,cli reported ClassNotFoundException.

       

      Flink SQL> use catalog myhive;
      
      Flink SQL> create function myfunc1 as 'me.aki.flink.flinkudf.MyFunc';
      [ERROR] Could not execute SQL statement. Reason:
      java.lang.ClassNotFoundException: me.aki.flink.flinkudf.MyFunc
      

       

       

      me.aki.flink.flinkudf.MyFunc is the identifier of udf,which defined like this

       

      package me.aki.flink.flinkudf;
      
      import org.apache.flink.table.functions.ScalarFunction;
      
      public class MyFunc extends ScalarFunction {
          public String eval(String s) {
              return "myfunc_" + s;
          }
      }
      

       

       

       

      after walking through the related code, I believe this is a bug caused by wrong classloader

       

      when using a hive catalog, flink will use  CatalogFunctionImpl  to wrap the function。 The

      isGeneric() methed  uses Class.forName(String clazzName) which will use a current classloader(classloader loads flink/lib) to determine the class。

       

      however with -j option, user jar is set to the ExecutionContext and loaded by another userClassLoader

       

      and the fix can be easy to pass a classloader to the Class.forName method.

      ClassLoader cl = Thread.currentThread().getContextClassLoader();
      Class c = Class.forName(className, true, cl);
      

      after do such fix and build a new flink dist,create function behaves right

       

      Flink SQL> select myfunc1('1');
      
      // output
           EXPR$0
           myfunc_1
      

       

       

       

       

       

       

       

      Attachments

        Issue Links

          Activity

            People

              akisaya akisaya
              akisaya akisaya
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: