Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
1.10.2, 1.11.2, 1.12.0
Description
with flink version 1.12.0(versions before also affected)
I started a sql cli with a hive catalog and specified a user jar file with -j option like this:
bin/sql-client.sh embedded -j /Users/akis/Desktop/flink-func/myfunc.jar
when i tried to create a custom function using class from myfunc.jar,cli reported ClassNotFoundException.
Flink SQL> use catalog myhive;
Flink SQL> create function myfunc1 as 'me.aki.flink.flinkudf.MyFunc';
[ERROR] Could not execute SQL statement. Reason:
java.lang.ClassNotFoundException: me.aki.flink.flinkudf.MyFunc
me.aki.flink.flinkudf.MyFunc is the identifier of udf,which defined like this
package me.aki.flink.flinkudf; import org.apache.flink.table.functions.ScalarFunction; public class MyFunc extends ScalarFunction { public String eval(String s) { return "myfunc_" + s; } }
after walking through the related code, I believe this is a bug caused by wrong classloader
when using a hive catalog, flink will use CatalogFunctionImpl to wrap the function。 The
isGeneric() methed uses Class.forName(String clazzName) which will use a current classloader(classloader loads flink/lib) to determine the class。
however with -j option, user jar is set to the ExecutionContext and loaded by another userClassLoader
and the fix can be easy to pass a classloader to the Class.forName method.
ClassLoader cl = Thread.currentThread().getContextClassLoader(); Class c = Class.forName(className, true, cl);
after do such fix and build a new flink dist,create function behaves right
Flink SQL> select myfunc1('1'); // output EXPR$0 myfunc_1
Attachments
Issue Links
- fixes
-
FLINK-22632 CatalogFunctionImpl.isGeneric should use ContextClassLoader
- Closed
- links to