Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
2.10.0, 3.2.0, 2.9.2, 3.0.3, 3.3.0, 3.1.2
-
None
-
None
Description
Hadoop aliyun module uses AliyunCredentialsProvider as default CredentialsProvider if user do not set fs.oss.credentials.provider. However, if user set fs.oss.credentials.provider to org.apache.hadoop.fs.aliyun.oss.AliyunCredentialsProvider, exception will be thrown like below:
java.io.IOException: org.apache.hadoop.fs.aliyun.oss.AliyunCredentialsProvider constructor exception. A class specified in fs.oss.credentials.provider must provide an accessible constructor accepting URI and Configuration, or an accessible default constructor. at org.apache.hadoop.fs.aliyun.oss.AliyunOSSUtils.getCredentialsProvider(AliyunOSSUtils.java:131) at org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystemStore.initialize(AliyunOSSFileSystemStore.java:154) at org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem.initialize(AliyunOSSFileSystem.java:344) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3302) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3351) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3319) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) at org.apache.hadoop.fs.contract.AbstractBondedFSContract.init(AbstractBondedFSContract.java:72) at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.setup(AbstractFSContractTestBase.java:177) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.fs.aliyun.oss.AliyunCredentialsProvider.<init>() at java.lang.Class.getConstructor0(Class.java:3082) at java.lang.Class.getDeclaredConstructor(Class.java:2178) at org.apache.hadoop.fs.aliyun.oss.AliyunOSSUtils.getCredentialsProvider(AliyunOSSUtils.java:125) ... 23 more
Because AliyunCredentialsProvider does not have corresponding constructor.
public class AliyunCredentialsProvider implements CredentialsProvider { private Credentials credentials = null; public AliyunCredentialsProvider(Configuration conf) throws IOException { -------------------------------------------------- try { credentials = (CredentialsProvider)credClass.getDeclaredConstructor( URI.class, Configuration.class).newInstance(uri, conf); } catch (NoSuchMethodException | SecurityException e) { credentials = (CredentialsProvider)credClass.getDeclaredConstructor() .newInstance(); }
Although the documentation says you should provide a `specified class must provide an accessible constructor accepting java.net.URI and org.apache.hadoop.conf.Configuration` if you set this configuration, we should make
AliyunCredentialsProvider compatible with this.
<property> <name>fs.oss.credentials.provider</name> <description> Class name of a credentials provider that implements com.aliyun.oss.common.auth.CredentialsProvider. Omit if using access/secret keys or another authentication mechanism. The specified class must provide an accessible constructor accepting java.net.URI and org.apache.hadoop.conf.Configuration, or an accessible default constructor. </description> </property>